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ABSTRACT ' * ' \ . . ^ 



Educational prganizations the learnincf process are extremely complex* 
This paper presents a conceptual framework for analyzing the complex relation- 
ships between school inputs and school outcoines and- fpr conducting cost-' ^ 
effectiveness studies in education. The paper discusses several concepts and 
techniques from the eureas of systems theory and economic analysis that *can be 
Vised as tools in an effort to improve the productivity »of the educational en- 
terprise^ In adc^^ion, the use of basic planning-progrctaming-budgeting proce- 
dures isi refcommendied to^ facilitate cost-'ef fectiveness analyses.' 

Several studies investigating productivity, in education are reviewed and, 
the ctnalyticdl problems in conducting cost-effectiveness studies* are ^xplored. • 
T^he paper points out the potential pitfalls in identifying, measxiring, and 
comparing progr^ costs and addresses the difficulties inherent in assessing 
program outcomes, Sever^ conceptual models designed for conducting cost- ^\ 
effectiveness studies in educatio^i are discussed ^d the results of a number ^ 
Qf empirical investigations emplojj^g cost-effectiveness techniques are pre- 
'sented. 

* The conqeptua^l framework developed iri this, paper views the education pro-^ 
duction process as a system consisting of four major componen'ts: (1) system 
environment and controls, (2) the^ scliool, (3) outputs of schooling, and 
(4) feedback* The first component includes socioeconomic variables that • ♦ 
have beM shown to influence* school outcomes and the policy f ramewoi^k within ' 
whi\^h sctefelp operate. The second component, the school, system itself,' , 
includes twQ ma3bt elements: (1) school respurce inputs — the hvunan resources 
(students and professional staff) and^the material resources (physical plant, 
classroom' equipment,, and isurricular products) / and (2) school resoxirce appli-* 
catiohs — the manner in which school resources, are combined or mixed to achieve 
designated objectives. The* outputs bf an educational system may be classified 
in several ways, e^..g., short-range and long-^ctnge, cognitive and nonccgnitive, 
or monetary and nonmonetary. The fourth component- is feedback based on evalua-, 
tion of the system? s outputs. / . * 
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INTRODUCTION ' . ' 

In the past ytree deca4es, education expenditures have ma4e up an ever 
larger compo*nent/6f the public budget, particularly at local and state govern- 
ment levels: ^p^^ increase in expenditures by educational' institution^ has \ 
been parti^2tilarly striking 6ver the past 15* years.. In 19^75-76, for exainple, 
^total ex^nditures *by public and non-public educational institutions 'dt' all 
leve>/totaled '$120 billion compared 'to a total of ?24.7 billion in 1959-60. 

Expenditures by* public arid non-ptlblic 'elementary and secondary schools 
increased from $18 billion to •$75.1 billion in this same 15 year period. The 
increase in expenditures by institutions of higher education has been even ^ 
more dramatic — from $6.7 bilTion in 1959-60 to $44 .-9 billion in 1975-76 
t (National Center for. Educational Statistics, pp. 189-90) . 

Accompanying the "^acowth* in education expenditures has come an increasing 

• concern 'for "accoi^tability. " As an enterprise involving the expenditure of 

• piiblic runds, accountability has always ^been a cpncern "bf edupators from a 

, fiduciary ppint of view, i.e., ensuring • tJ^at public funds are not misappro- 
. priated, lost, or stolen". ^In recent years J ^however, the definition of account- 
ability has broadened to encompass concern ^for achieving the most efficient use 
of the resources devoteH to education. This concern/ is perhaps .bjest expressed 
by the question, "Are we getting ouV^.money's worth from expenditures for educa- 
tion?" - " 

Given the' magnitude of expenditures for education,, it is understandable *^ 
yiat citizens and policy makers question the productivity of educational 
institutions -and express concern for what they perceive as 'inefficiency in 
,^^sing resources demoted to education. Haggerty (1973, ppOl-10^ pointed out- 
that in 1970 only four- percent of American workers wfere engaged in agricul- 
ture,' forestry, or fisheries as. compared to 40 percent in 1890 and 20 percent 
. in 1930. Yet the four^percent in 1970 were producing substantially more food 
and forest products than the 40 percent in 1890. * Based on these data,; pro- 
ductivity per person engaged in agriculture ijicreased more than .ten-fold in 
that 80-year period.^ But on the othfer hand, Haggerty pointed out that between 
i;930 an& 1970 our population increased by about 60 percent, *our school en- 
rol*lment 'increased by about 200 percent, a!nd the numbe'ir of persons employed 
in education increased nii|^e than 400 percent! . Given these data, the interested 
citizen i's likely to conclude that productivity /in education decreased during 
the 40*years between 1930 and 197 O' and that the ^naJLity o'f today's education 
may even have decilined. This point of view is likely. to be reinforced by 
widespread publicity currently given dec lining college board and other 
standardized te§t scores and the desultory result^ reported by tl^e National 
Assessment of Educational Progress. , - ♦ 



^^learly, the question of whether or not the resources allocated %b educa-^ 
tion are being used efficiently is of gr4at ^importance to s'ociety* ^e ooncern 
for achieving greater ef f icienc^^ has been, demonstrated over the past decade, 
by the application of- analytic techniques, such as planning--programming- » 

^budgeting, input-output analysis/ cost-benefit analysis, and cost-effectiveness 

. analysis, to- educational problms^ Unfortunately, too many analysts have ^ 
sought simple answers to- complex questions^ They have frequently failed to 
recognize the extraordinary coinplexity 6f the processes 'involved in, human 
\eatning. To a^itain greater efficiqncy^ and productivity^ one must be .able 

* t^ specify the resources needed, the quantity, and the combinati-dn, to produce 
a desj.red level of knowledge, \kill,' or behavior/ Educators .are confronted 
with these kinds c^f questions -A Do differences in the kind and, amount of 
resources lead consistently to different educational outcomes? What school 
characteristics, conf igxirations, and programs lead to differences in outcomes? 
What impact* do sch ools 'h av^ on students? Because schools are not ideal 
• laboratories in which organizational features (suchAs the distribution and 
utilization of resources, con5)osition of the' student body, placement of 
sttidents in program^/ methods of instruction and- e'^aluatibn, or operational 
objectives of school programs can be manipulated), adequate research ^evidence 
is difficult^to obtain. Despite these difficulties, hctwever, educators must 
ask whether the resources ^t> their disposal are being effectively and effi-^ 
ciently allocated and util^^ired. ' • ' 

^ ^ .We ^o aiot propose in this paper to identifjr cfuicjc ^d easy w^s to [ 
improve productivity and eff iciencr^^n the use of resources , devoted to 
schopling. We~are primarily conc4rned with- developing a conceptual frame- 
work to facilitate analysis of the-etS^l^ relationships between school re- 
souJTces, schobl processes, andT^school out6omes. Application of systems 
analysis and economic analysis will be discussed, selected literature bearing 
upon productivity and. efficiency in education vill be reviewed, and a concep- 
tual schema designed to guide the search for connections between school re- 
sources, school processes, and school outcomes will be formulated. 
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— V ' gDUGATION AND SYSTEMS- ANALyS 16 ^ 

' ^ . ~^ ' ^ ' ' » > 

The concept of a system is not ^ reeent one and the word' "sys.teitC' has 
many definitions. Kershaw and McKeari (1959,. p. 2)\ in their pioneering work. 
^.applylng» 'systems analysis to education, denned a system' as "a set^ of inter- 
^ related factors th'^t "are-/used together to produce anotftpjiit." Martin (1969, 
p. 49) ^d^f ined a system as "an assemblage or combination of things or parts 
forming'.a complex' wtiolis^" He noted ,tbat onef of the ^ost important^ oharapter- 
istics of a system is'that it is compos^ot a hi^archy of subsystems.^ 
Bar\g1iart (196^, p. 26) defined a system as "an integrated assembly of inter- 
acting elements, designed to carry out cooperatively a pr-e^Jetermined function. 
These definitions are siinilllr. l!ach converyB-^he notion that a' system is com- 
plex, that It is composed of several elements, that* its elements are inter-. - 
active, and that* the elements arid their intpractioi) are purposive. Under any 
of these definitions, a classroom, a school, a' school district/ all of the 
school districts in a state,' &r all of the school districts in' the nation 
qualif:^ as systems. , ' • ' - , . ^ 

^ Although the study of systems is not new, the emergence' of systems 
^alysis as a SRgcialize;i discipline is relatively recent. .Systems .analysis 
is concerned with t^e careful, disciprined* stuAy, of systemsNio identify 
feasible, ef^iciSvt^^and acceptable^means for 'accpinplishing a given purpose. 
As Hare (1967, p". 2jTtas ppinted but: 

• To understand and imprOve . . . systems by utilizing the rela- 
tionships that resent when diverse components ara. combined, in- 

, vestig^tors must be able to follow and ^exploit the chain of work • « 
flow, informatipn flow or material flow — the. connecting links 

- that tie the systerjs- together . And, lAese connecting 4Jjiks al-. 
most always lead across the boundaries of educational discipline 

• and organizational function. » , ' - ^ . 

$ ( ^ * " ' 

Systems analysis may be described 'as a comparisQn of alternative means 
of carrying out a function when the means 'are complicated and include a 
number of interrelated elements. Before a system can be analyzed/^a model 
of that system myst be conceited. A system ipodel* has i^een described as "a 
wide-angled lens trained on an organism so that it can 'be seen in its. en- 
tirety, including the relationships among its (component) parts and between 
the organism, and the environment (Coombs, 1968, p. 8]-." * 

* The model of a system provides the conceptual base for analysis of *that 
system. It must represent as accurately as possible the activities associated 
with the system «nd their interrelationships. -Such a model typically will 



include "a set of inputs, which are subject to a process, designed to attain 
certain' output;s, which are in\:ended to s^ttisfy 'the system's objectives 
[Coombs, 1968,^ p^» 9} ♦ " A cu^^iculaf innovation in education, for example, 
may entail 'significaift changes in instructional xftethods which, in turn, re- 
quire changes in the" utilization' of time, -physical facili<ti^ and equipment^ 
required, and the' number and kind of personnel* needed* Alteratioirs such as 
these Jiave consequences for the systefti's input Requirements, for the nature 
of . the' process employed, and for the quanti^;ty and quality of the system's' 
outputs* , ' ^. ' . / , -(P ' 

Xershaw and Mckean (1959),- in. their work for the Rand Corporefcion , pro- 
posed applying syst'ems analysis tecl^iques to assess the possibility of 
xfaking quantitative comparisons of educatilnal systems* They stressed ~ETiok^^ 
ne^d for conceptual development qf^ tno&els ^Ind/or relationships reqyiired to ^ 
estimat«%aH costs, altenjative processes^ and output measures heeded for 
cmalysis of educational systems. They discussed the difficulties involved, 
in measxiring educational output proposed a procedure for estimating irn- 
cremental costs and ^utput from new systeigs, and suggested a method foif 
interfacing cost and outpjut data to, cpmpaure systems. Although vcirious appli- 
cations of systems analysi^s t/soimiques in education could be cited, perhaps 
the i3est known hav^ been those associated with attem|>ts to develop program 
planning, and budgetj.ng systems for educational organ^izations . 



. \ ' til, 

EbyCATT^N ANB ECONOMIC ANALYSIS^ 



EcJonpmics dei^rls wi^th allocation and -utilization of scarce resources, i 
i.e*, resources for which /Shere exist alternat§.ve uses. The economist seeks ^ 
to allocate tl^pse scarce resources among alternative uses in a, manner which 
maximizes the satisfadtions gaisfed by consumers; ^ For the economist, maximum 
efficiency is achieved by utilizing resources' i-n such ways that any change 
in their allocation among alternative uses^ vould reduce. the total satisfac- 
tiori'of -consumers. Since the resources allocated to educartion ♦by society' » 
could be devoted to many other uses, 'the most efficierlt use of •these resources 
is a legitimate concern of both economists and Educators, . . . 
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ION AS AN ECONOMIC GOOD 



Although A^am Smith and bther eart.y, writers Suggest^Si links between '^ ' 
education and economics, it is only in regertt -.years that economists have 

^ devoted serious acttention to education. Th^ growing i^lterest in the econoitvic 
aspects of education can he attributed to recognition that (1) education is 
one of our largest enterprises and' a major employer 'of skilled per3onneJt# 
(2) ediTcation may influence significantly an individual's, ^ploytnenti and 
income opportunities and thus affect' tl\e distribution of income and wealth 
in society, and (3») education,, as the primly provider of trained and skJiied 
personnel, is an important' factor in economic growth and development ^ 

, (O'Donoghue, 1971),. / > 

! ^The U.S. economy i,s qften deferred to as a "market economy" in which the 

flow of 'goods and services is regulated* by the d^apd for them. The term' 
demand, as used in economics, refers to a fymctional relationship between the 
price' o^^specific goods <5r services^i^d the amoui^t/Of those? goods or serviced 
which will be purcbaared. Demdnd is a subjective concept only to the eiitent 
that it measures the relative value conslim^rs place upon given goods or .ser- 

Jvices in comparison with other, goods or services that they could buy. The 
demand for most goods ^artd services provided through 'the operat*ion/ of the 
market can be measured objectively and quantified .with precision.^ As the . 

, pyice of Specific goods or services increases (assuming the demand* for it 
^ is elastic), the. amount purchased will generally decrease, because other goods 
and serviced wilL become more attractive to the consumer. Thus, when the 

^marlcet is operating freely ( uncons tr ain e/i- by monopoly, regiilations, •etc. ) ,^ 

^the AoW of ^oQds and services is regulated in^ a manner which reflects the ' • 
r fej^tive aememd for those goods. , ^ . > . 
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The component, of 'the t6tal economy in which thi^arket serves to. allo- 
cate goods and services in accordance with the relative Supply and demand 
for them is generally referred to as the private sector.* Although the pri-» • 
'^ate sector makes up the largest portion of bur economy, a significant 
amount of gopds and services is provided ^y agencies of government, i.e., 
the public sector of the economy. The supply of most goods and^ services' 
provided through the public sector — highways, national defense,- police and ' * 
fire protection, "Education, for example — is determined by 'legislative and/or 
administrative 'judgment with regard to the amount of ^ given good or service 
that should be provided to best' 5exve the general welfare of society. In 
the public sector, the /judgments of ^-public off icia.1? (based upon their per- 
ception of. "needs" rather than the operation of tlje market) determine th^ 
kind and^aipDunt of goods and servio^s^made available to consumers. , - 

Ai^^ One of the serious problems encountered . in a "pure"* market' economy is' 
that individual consumers invariably act only in their own self- interest, 
wfiich is not always the best interest of the total society. Application*, • * 
h'f the concept of need enables public officials to decide 'upon the, kind a^ . 
*^ouAt of services, such as education, national defense, police and fire pro- 
tection, and highways, which they 'believe will best serve the general public. 
Society's need for educational 'services for handicapped children^ for - ^ 
example, may be far greater than the private demand . for such service as re- 
flected ^by the market.. .Because educational services for handicapped children 
often ate very expensive, the parents of handicapped' children frequently are 
not in a financial positioh to buy the services. Public officials, however^ 
may decide that the general welfare of society will best, be served by ajjpro- 
priatihg' public funds to provide special educat^.on services for handicapped 
children. In this way the concept of x\4e6 rather than the concept of demand 
is. employed to determine the level of goods and ^er\>ices provided through 
the public sector of the" economy-. • , ^ ' 

EDUq^TION AS .A MERIT GOOD ' ' ' 

MusV^ve (1959, pp. 6-15) distinguished between two classes of goods 
which typically are provided through ^the economy's public sector--sociai' ' ^ 
5[^ds and merit goods. He defines social goods as those which by their T^ery 
nature cannot be supplied effectively by ,the market because* they are equally, 
.available to*a^ll persons whether or not they pay for them. National defense * 
and the judicial system are examples of social goods.. Merit goods are de- 
fined as those* goods which could be provided through the market but are 
thought to be so important to society that tlieir' provision cannot be l^ft • 
entirely to the 'unenlightened self-interest - of individual consumers. Educa- 
tion is a prime example, of a iqllrit, good. The mariiitenance of an educational 
system in which all citizens have free access '.to at le'ast a mini;a\am level of 
education is thought to be vital' to democratic self-governmenjf . Therefore, 
education is provided primarily through the public^sector and financed pri- 
marily through the publiq budget. . - ' 

Anot^ier reasdii exists for financing education 'through the jjublic sectpr. 
VSiile^ education yields direct benefits ta students a.nd their families (as 'in 
inpreased earning potential) , it is also beli^veti to yield benefits to the'', 
community and to society at large (for ex^le in the^ form of increased pro- 
ductivity and lower welfare ^ costs) . Benefits that accrue directly to students 



are termed internal or private benefits; those that accrue to others in ^ 
society are termed external or social benefits,. Externalities (ext^ern^l 
costs or benefits) arise when goods or services either confer benefits or 
impose costs on persons other than the consumer or the producer. Educatiorx 
is generally thought to be characterized by substantial externalities be- 
cause it affects so many people who do not buy at, directly 

The importance of external benefits lies in the fact €hat private deci- 
sions concerning whether or not to purchase education will be evaludtec^ al- 
most entirely on the basis of internal benefits, those benefits^ gained" * 
directly by students or their families. If education were supplied ^only 
ChrougVi the private sector/ the decision to invest^in education would be 
based solely on the anticipated direct, benefits to the student and would dis- 
regard any external ben^its whicli may be realized by society. 

While it is difficult to place a precise value on the social beriefits. • 
associated^ with education, they" cire not insignificant. The social, benefits 
of education incljade greater flexibilityand adaptabili-ty of' the labor force 
which/ in turn, enhance our capability to develop ancj apply technological im- 
provements. Education is a majo^: instrument promoting equality of opportunity 
in society and is generally recognized as a requisite\for successful democratic 
self -government. Negative externalities, such as higher unemployment and 
-crime rates, may result from lack of education. 

The external benefits associated with education provide a -persuasive, 
case for public support of education. As we have noted, externalities are 
no t^ considered in private investment decisions and, because they are ignored,- 
the optimal allocation of resources to education l?y sipciety would not be 
achieved if education^ were prc^vided only through the private sector. Public , 
financing of education, then, provides a mechanism for giving due considera- 
tion to jaQucation's external benefits in the process of allocating resources. 

It should be emphasized that there is no economic reason why education 
could not be supplied entirely through the private sector. Consumers (house- 
holds) could purchase education from privately operated schools just as they 
now purchase many other goods and services. However, the 2unp\int of education^ 
which would be purchased Xfi ktlvb marked by households probably would be con- 
siderably less them optijpe^in terms of the general welfare of society. Thus, 
in the United States we mave chosen to* finance education prim^orily through the 
public budg ets Th e difference between what consxamers (househblds) would spend 
for edytfation if*i^ScoMd 'be purdhasejj only in the market i.t full cos^. and the 
total amount of money^expended for education in* the private and public sectors 
combined may be viewed as a .form of ptiblic subsidy* The purpose of this sub- 
sidy is to ensure that at least a minimal -level of educational o^qrtunity 
is provided t«^ll childr^. ^ v . 

It is important to note that in the p3;ivate sector the supply- of goods 
or services is <^term*ined by tHe market which, At least.* theoretically, will' 
• .e^erve to "weed out" inefficient producers, i.e^* those whose cost of produl::- 
tion exceeds the going pric^ of the goods or;, services. " In the public seotor, 
on the other hand, supply is determined lay the operation of* a political 
decision-making process, not by the operation o£ the market* Thus, in the 
pxiblic sector, there is np mechsmism to enst^re that goods and services cire 
provided in thfe most efficient possible manher. The lack , of a mechanism to ^ 
automatically "weed put'V inefficient programs undoubtedly is one reason for 
the growing interest in the evaluation of programs iVi the public sector. 
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EDUCATION AND THE CONCEPT OF HUMAN CAPITAL 



'^^ The concept .of humai> capital is bal^d dff^ifhe id*'a^£hat the skills and 
^ knowledge possessed l^y people are,, in fact, resources^and that these resources 
represent a ve^ry important part of the stock of capital avail^le-to society.' 
From an econoidic viewpoint, 'capital is characterized by its ability to generate 
future satisfactions, future earnings, or )Doth. Education qualifies for con- 
sideration as capital just as land^ industrial gtS^ants, or coal mines do be- 
cause education has the ability to generate future satisfaction and future- in- 
come. Viewed in this way, the economic val\;e of education can be expressed as 
a function of the income stream that a^gi\ren level of education is able to 
generate. ) ^ 

When education is viewed as>^ form of capital, decisions with respect to 
education, whethe*r made by j| student, a household, qr public or private 
agencies, are dealt with as^invei^tment decisions and are based on the relative 
rates of return available* to alternative investment opport^ities. Education 
is viewed' as one component in the total s^ock of capital. Investment decisions 
^abqOt education follow th'e rules which guide other investment decisions. Thus, 
additional investment in education occurjs only if the rate of return from the 
investment equals or exceeds the rate of return available from alternative in- 
vestments.-^ 

Economists have developed a number of- analytic tools t9 evaluate alterna- 
tive investment oppoi*tunities. Although these tools were devised to evaluate 
alternative fnvestmerfts in physical capital, they also can be applied to . ' 
analyze costs and benefits associated with human capital. Two of the primary 
procedures used are present value analysis and "analysi^qf internal rate of * 
return. ' / * ' " . 



PRESENT VALUE ANALYSIS ' 

T^e costs of obtaining educatibn are incurred over one period of time; 
the* financial benefits^ which acc^rue as a result ^f that e5Jucation are derived 
over anothei:* period of time. To f^cilit^te comparison of a s\ream of inyest-A. 
ment and a stream of ir^come', it is useful to' reduce both streams to. a base 
year value; This is thfe^basis for present value .analysis . As Thomas (1971, 
p. 22) notedf 

Present value analysis consists 6f using compound interest and. 
compotind discount procedures to reduce a 6ti:eam of cost and k 
' stream of income to their value at a given base yfear. A dollar 

received todayl could be invested to yield interest; if that / 
^ interest is-^alTowedf to accumulate^ today's dollar will^ increase /* 
at a compound rate of interest, B\^t this also implies that a . ' / 
•guaranteed payment of one dollar in ten years time is equivalent / 
* t^o a smaller amount of money today. ' , / 
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For further explication of this topic see Schultz, ^1970, pp. 29-57. 



If the ratte of interest is i, the present valuer, of one dollar 'to, be received 

• ten;ye/rs in the future can be expressed by the formula ^ . Generalize 

. ' Jl+i)^ . ' 

ing, the present value of an income stream of Y dollars per year for n years 
^ginning with year t\ can be expressed by the formula: * ~ 

/ ' .i ■ " ■ , 

fe" VI- 
VO (Y) = ^ 



t=l (1+i)^ 



Sinj'ilarly,tthe /resent value of a stream of cost the ambunt of X dollars 
per year'. can" bef expressed by the equation; / 



( 



t=n 



Vo(X) = 



Xt 



t=l (<l+i) . 

\ - 

Usxng thiS approach, investment in education occvjrs if the present value of the 
additional benefits associated with an increment of education is greater than 
the present value of the additional cost incurreci in obtaijQing thSt increment. 
Thus , em investment would be undertaken, if . ' 



Vo(Y) -Vo(X) 



t=l 



Yt 



(1+i) 



tr=n 
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Xt 



''(1+i) 



> 0. 



Obviously, the choice of the djs'count rate to be apjrlied is cri^tical in 
determining wj^ther or not additional investment Jji education should be 
undertaken, -fflfe choice of tha appropriate discount rate is essentially a 
subjective decision. If the desire is.'to receive income as soon as possible, 
a high discount rate will l^e utilized, thus reducing the presej<^; value of a 
future income stream. If, "on the other hand, one is. willing to defer receipt: - 
of income, a lower discount rate will be used, .thus increasing the present 
value of_a'future income stream. In determining the appropriate discount 
rate, SMch factors as the time preference of the individual, rates of return 
available on other investments, and the iriterest rate one n^ust pay on borrowed 
funds are considered. 

RATE OF RETURN ANALYSIS ' ' ' ' • 

. < ' . . ' 

The xnternal rate of return is that rate which equates .the present value 
of the investment to zero, as shown by the following equation: 



t«n 

Vo(Y)-Vo(X) = 2 
t«l 



Vt 



(1+i) 



Xt 



t=n 
t=i -fx+i)^ 



As stated -by Thomas (1971^ p. 24), "The decision rule is that ■ individuals " (or 
social groups) "should continue to invest as long as the rate of return exceeds 
that obtainable from other alternatives which are considered." ' V 



, Pj^6sent*'varufi analysis'* and rate^of return analysis can provide useful 
infojfttpation to policy-make'rs concerning macro-level policy decisions. .That 
is> i*t is pJo^sible to evaluate returns to additional investments in edxlcation. 
compeared with, returns, to additional investments in other areas, such as public" 
health/ highways, qr. flood controlt Informatioa obtained from present value 
or rate't?f rfc-turn analysis can-^also be of use^ to individuals considering whether 
they ,shQuld invest in additional increments of education in comparison with 
other investment o^jportunities., \^ ' ' 

These techniques do not, howievej^ provide guidance or direction with 
regard to questions concerning 'how, tQ most efficiently use resources allocated 
to education. That is, they are "not particularly useful in micro-economic 
analyses dealing with the question*'of ' how to maximize productivity of re- 
sources devoted to education "once, a level of investment has been determined. 



EFFICIENCY AND PRODUCTIVITY IN EDUCATION 

In the remainder of this paper we ^Ij^ll be concerned with tfie brq&d* issue 
of ^optimization. That is, how Qan the resources allocated to education by 
society be most efficiently used? We assume that the level of publi^ in- ' '\ 
vestmeiit in education will qo^trinue to be determined through^the political ^ 
dfecision-making process. We also assume that individuals -will continue to 
make their own decisions With regard to whether or not«they will irtvest in , 
additional increments of education^^ 'Thus, the level of resources* avjailable 
for education will be. a function of , both public and private decisions. Given 
any established level of investm^t, the question. confronting *educat;oi^ is how 
to use available resources most efficiently, i.e., .^ow to maximize the desiired 
^utputs- of the educational process from a given J.evel of resource .input. 
V , Although input-output, cost-J)enef it, and cost-effectiveness analyses have 
iJ^ea undertaken in the} field of q^ucation, the studies in which these tools 
have been employed generally have utilized data aggregated at a relatively 
high orgctnizatior^kl level, siiich as a school -system., a state, or evein the . ' 
nation We shall be particularly interested in examining the application of 
these .tools in djpaling with questions of sub-optimization. That is, w^-are 
interested in the possibility of using more discrete unit9,,gi/f ctnalysis in am 
attempt "to deal with questions concerning the most effici^t resource alloca- 
tion for individual schools or groups of students possessin*^ pgecified 
characteristics. We are concerned that despite widespread recognition that; 
each student is a unique individual, previous cUialyses -have treated students 
as if they were inteirchangeable units. 'We al^ are .cbncerned that the.. praCtide 
of aggregal^ing data at a school district level inevitably destroys the richness 
of detail and variability observed, when individual students are the basis for 
analysis. We believe thAI: -foc.using upon sub-optimization in the use of resources 
holds greater promise in providing useful information to school l^vel decision 
makers. In fact, we are incli'ned tb believe that full optimization in the 
utilization of, resources allocated to education cam only be, achieved throu'gh 
successive apptoximatlons based on sub-optimization of piroductivity at speci- 
fic educational levels pr for specific target groups.' 




IV . • 

school' PRODUCTIVITY-- RELATED RESEARCH FINDINGS'/ 



• • Numerous studies "^f school productivity and effectiveness have been 
conducted .over the past .fifty years, the initial "cost-quality studies" 'con- \ 
. ducted' by Mort and others examined gross relationships between aggCfecfalfe 
.levels of educajdonal expenditures and various measures of school processes 
and outtc^es (Mort Reusser, and Polley, I960). While these early studies 
qonsistently demonstrated tjiat high expenditure^ schools Were more effective 
.than low ex^nditure schools in terms of the criterion varikl>les employed, 
they did not reveal which school inputs ihave the greatest i^ct on student 
learning or even which Resources make a difference/ In short, the studies 
provided little, if any, .direction with regard to how to allocate available 
school' resources effectively and efficiently. ' . • 

Over the latet "^two dicades a variety of measures assumed to represent 
proxies for inputs i:o educational processes have been used in an attempt to 
determine school effectiveness'. In one of the first input-output analyses, * 
Mollenkopf and Melville (}f956)^ surveyed a sample of approximately 9,500 ninth ^ 
•grade students in 100. schools and 8,400 twelfth grade students in ^06 schools* 
The researchers used questionnaires completed by' school principals to develop 
^an initial list^of 34 inde^pehdent measures, including variables such as the 
nature of school facilities', \the degree level of tffe principal, the numl^er of' 
special staffs the pupil/teacher ratio, a drop-out index, and the average 
teacher salary. Student scores on seven different aptitude and achievem^t 
tests were used as dep^dent measures. Attempting to control statistically for 
socioeconomic factors (such as Occupation^ of .father , size of commuip.ty, and per- 
centage of support from state^ aid) and employing stiepwise multiple fegressioti - 
techniques, the authors obtained significant relatdonships between'^^h^ir ^ 
meastares of student achievement and (1) per pupil instructional expenditujres,; l 
(2) number of special staff^ e:g», sjchool psychologist, reading specialist, * 
and guidance co\inselor, (3)" average class size, (4) pupil/teacher ratio, and 
(5) percentage of the school's graduates^ entering college. 

'Three years later Goodman (1959) conducted the Quality Measurement ,Pro-* 
ject (QMP) which analyzed a sTample of 70,000' seventh and eleventh grade students 
drawn from 102 school districts in NeW.YQrJc;sta*«^. After controlling for ' 
socioeconomic factorl/ Gb;)dman found a significant relationship between the 
achievement of sieventh Wde students and *(1) per pupil instructional <exp^n-^ 
ditures and (2*) number of" special staff per 1,000 students. \ At, the same- t^ime, 
two additional, factors were linked to student perform^ce—teacher experience • 
(number of teachefs in a district with five or more years aa classroom instruc- 
tor) and "classropm atmosphere" ' (an observational rating att^pting to measure 
the degree to which a teacher was stiaident oriented) . Reinforcing the findings 
Of the earlier^Mollenkopf-Melville wW, the QMP study also suggest9d that 
certain attributds of school personnel are -likely to play an in^rtant role 
in the learning process. ^ \ 
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Thomas (1962) utilized Project TALENT information^ and 1960 census data 
to examine th^ impact of a large number of home, school, ctnd community vari- 
ables on student achievements Thomas' sample consisted of tenth and twelfth 
grade students from 206 high schools in communities with populations between 
^ 2,500 and 25,000. The schools were 'scattered among 46 states* Using stepwise 
multiple regression .techniques, he found statistically significant relatiQn- 
ships between student achievement and (1) J^eginning teacher salaries, ,(2) teacher 
experience, and (3) number .of voluijies - in/xhe school library* 

Benson (1965) corlducted an extensivfe study of California's public schools 
in a964\ ^ Data were obtained from the 1960 census as well as from statewide y 
and school district records to study a sample of fifth grade students in 249 * 
*■ school districj:s. The sample wa^ divided into three groups based on district 
size (number of students in average daily attendance) ♦ Benson reported th^ ^ 
-teacher salaries and per pupil instructional expenditures were significantly 
related to- student achievement (as reflected by test scares in reading atnd 
mathematics)/ In addition, Benson found that in the middle-sized districts 
(2,000 to |4,500 pupils)', the salaries of administrators were associated posi- 
tively wi^ student achievem'ent. ^ , • , 

At ^S^same time that Benson was conducting his study in California, 
Kiesling (1967) was re-examining data collected earlier in the New York QMP 
Study • As in the Benson study, the QMP ^J^k utilized data aggregated by 
school districts rather than individual schools as the unit of analysis* 
Kiesling also found that iper pupil expenditures were associated positively 
with student performance. .The relationship was quite strong ^in uirban school 
districts but weak in rui^al school districts. ^Kiesling observed that an 
additional expenditure of $100 was associated with 2.6 months of achievement 
gain at the low end of thp expenditure range and with 1*4 months at the high 
end of the range.* In addition, Kiesling found that; .school district size and 
student performance were not related. 

The Equality of Educational <(pportuniiy (EEO)->study , jtopularly known- as 
the Coleman Reports (Coleman, 1966) , was the first: Id^ge^scale input-output 
sWvey of the nation's schools. JEfhis landmark stucjy'^^l&veyed 645,000 
students' at various grade levels in 3,100 schools. Input measures con- ; 
sisted of 93 variables grouped into fciir major blocks — home background 
factors, teacher characteristics, stuc|ent-bodY variables, and school facility, 
and curriculum measures. Scpres on standardized achievement tests served as 
©utput measures. ^ ^ . v 

The rese^ch by Coleman and his colleagues highlighted t:he Significance 
of 'the relationship between socioeconomic milieu of the school emd student 
performance. Home background factors were clearly the most important ^roup 
, of variables explaining variance in achievemealt levels rfor all four major sub- 
groups of students — sopthern and Northern biacjcs ^d southern, and northern 
whites. Student-body characfter is tics, €^.^., plans to attend ''college, school, 
attendance, and racial composition, were the second most important group of 
variables explaining variance, particularly in ^he achievement of minority 
students. Of the school-related variables, none of which accounted for a 



^Pro^ect TALENT surveyed a^ nationwide sample of approximately 300,OjOO students 
in almost ^1,000 high schools in 1960. Thfese student's provided detailed in- 
formation about themselves and also complet€?d aptitude, ability-,* achieyement, 
and interest tests. For additional information regarding this national' datia 
bank, see Flanagan,, D^vis, D^ley, Shaycoft, Orr, toidijerg, and N^yman, 1964, 

^and ShayoDft, 1967 . ^ \ , ^ I 



^ large amount of .variation in achievement, teacher characteristics. had*the 
greatest impact, onpe^gigain affecting minority students mos^. 

To say that these findings generated considerable controversy woiiid be 

' ail uijderstatemept,^ Many r"esearchej?s^ere unwilling -to^accept the proposition* 
that school rasonr9es. hard little or no': effect on academic achievement. Critics 
of Coleman ''s work suggested that the relationship between school resources .and 
acadehiic achievement had been substantially understated because of defective 
measurement of school resources/ inadequate control for social background, 
and inappropriate statistical techniques. 3 The quality of the Coleman data ^ 
was questioned, particularly because of the apparently high jporrelq^tion 
between family background factors and school, resources. Because of this joint 
variance, the explanatory * power of * school variables was drastically reduced * 
simply because environmental variables were entered into the regression first. 

A number of Researchers h^ve re-analyzed the EEO data and, although ham- 
pered by limitations of the original data, have -been able tb clarify some of 
the problems involved in applying the production function concept' to the 
learning process. In one of the first re-analyses, ^Janushek (1968) developed 
a conceptual model to estimate educational production functions for black and ^ 
white ^ixth grac^^ers in northern metropolitan schools. The results, of Hanushek's^ 
estimations disclosed that'teacher characteristics, . such as verbal ability and 
years of experience, were significantly related to student achievement. Thifc 
finding was of particular interest sinc^' the Coleman da^a oh school .variable^ 
permitted an investigation of inteischool variance* in achievement 9nly and not 
intraschool variance which was considerably 'larger. - ' * - 

Bowles (1970) presenl^ed a compirehensive-, treatment of educational produc- 
tion functions in his re-analysis of a subset of the Coleman data consisting 
of* twelfth grade black tnale students. Bowles* work reaffirmed the importance 
of teacher characteristics and sjlaggested tl;]iat certain other scihool inputs were 
important. These other schpol f actorsi incluteS the average amount of time a 
teacher spent in guidance activities am the number of days .a school was in 
session during th^ school year.. Bowles argued that student character istics" 
such as attitude and motivation can be viewed as.eithet inputs ^to- or outpiks 
of the learning process^ He tljen developed a model involving the solution of 
a s-et of simultaneous equations to determine the Relative effects of the ' 
related variables. ^ ,> ' 1 * 

Also uli^lizing the EED data. Levin (l97pa) examined , a sample of 600 whit;e 

^ sixth grade s€lidents drawn from 36 schools in a, large northeastern city. Levin 
obtained statistically significant relationships between%tudent achievement, 
teacher experience, aJid quality o^ undergraduate "institutions attended by^. 
teachers. Levin also pointed out that' some factors affecting sttiaent .achieve- 
ment are^ at the same time affected by achievement. To investigate^this \nter- 
activie process. Levin developed a conceptual model to? illustrate jj^e inter- 
dependenfce of student achievement, student *motiVation, student efficacy, and 

^parental attitudes. He also presentecf ^a. methodo'logy for "solving the complex 
system of simultaneous equations that differs from the^ techrilque employed by 
Bowles. * ' 



^See, for example, Bowles and Levin, 19.68a, pp. 3-24. Also see Bow;Les and Levin, 
1968b, pp. 393-400. For a comprehensive critique of the Coleman Report, see » 
Hosteller and Moynihan, 1972. ^ \ 
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B^uAhead and his colleagues (1967) examined -input-output ^relationships 
in public secondary schools of two large-city systems* jrfteir saii\ple included'^ 

. 39 Chicago schools enrolling approximately 90,000 students and 22iJVtlaftta 
schoals enrolling about 19,000v students. The researchers used d'ara obtained^ 
frpm the 1^960 censug and from the two school' districts to conduct separate* 
analyses for each city. In addition, they ^compared the major findings in 
tiiese two cities with a sample df 177^ public iiigh school?' locateS in smaller 
communities. Information these latter ^schools was drawn from the Project 
TALENT data bank. • ' . V * 

Similar (but .not identical) independent and dependent measures were used 
for each of the three subsamples. The authors considered ther impact of 
variables such'as median family incom.e, average' daily attendance (ADA), age^ 
of school building, teacher experience, ^d per pupil' exg^diture on school^ 
outputs such as achievement scores, number of dropouts, and< students' P9st- 
high schooj intentions ♦ In Chicago, highe:; family income levels ^and newer 
school Ipuildings were associated with lower dropout rates. Family inpome 
was also found to be associated with IQ and reading scores. Of the school 
variables, only teacher experience was related to student xeading scores. 

"For the Atlanta schools, family income was ag^n related to dropout rate anfi' ^ 
achievement s.cores. In addition, a low rate of teacher turnover was positively 
associated with higher test scores. For the h%h schools io smaller communities 
the authors reported a relationship between reading, scoifes, and (1) family inT 
come, (2) a^e of school building, (3) teacher experience-, and (4) beginning 
teacher ^ salary. - * * „ 

Severar^ input-output studies in individual states or^school >distri^s were 

5 published in 1968. Katzman (3r968) used cross-secUionaJ. data from 56/^ementary 
schools iri' Boston to examine the importance of home background* factors and ^ 
school variables in explaining change in student achievement , between second' 
and sixth grade. Usin^ a stepwise multiple regression technique, statistically 
significant relationships were obtained between gains in reading- scores and the 
percentage of students in noncrowded classrooms, the nuiober of students in the 
attendance area, and the percentage of teachers with l-10*years"6f teaching • * 
e3<fperience. in addition to providing further evidence that teachers affect ' . 
pupil performance, Katzmcin also pioneered the use of several measures of 
school output in his s*t.udy, including such jioncogni1>iv9 outcome indicators 
as school "holding' powfer" and student "aspirations."^ ^ ^ ^ 

Cohn (1968) investigated input-cJutput Relationships in 377 public Kigh 
school districts in Iowa. He utilized data from the lo^a ^tate Departmeht pf' 
Public Instruction to examine possible economies of scale in school district ; 
operations. An attempt was made to control statistically for. geographic and 
population differences withra set of eight school district variables s.erving 7 
as the input measure. The output measure was b^sed on the gain in student ' 
achievement ^scores between tenth and twelfth grades. Employing ir)ultiple re-, 
gression techniquesv/Cohri." foOnd that higher teacher salaries and fewey dif- ' ' 
ferent teaching assignments were associated with larger growth increments in \, 
test', scores. Cohn also estimated the optimal school size for Iowa to be abquti 
1,500 students in ADA. ' ^ ^ « 

In a, similar study, Raymond (1968) examined the quality -of schaoliiig in 

/West Virginia, ^ymond's sample consisted of approximately 5,000 students 
who'ent^ed West Virginia University between 1963 and 1966.. These student^^ 
represented, 49 county schooj districts in West Virginia. The inputs^ included 
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five population characteristics, %our school variables, and six^fe««C!Ter salary 
variables. Output measures cS^tsisted of f teshman ^^^^ep^i^^^^^^erages and 
American College"* Test scores, ' After grouping students^y counties, -Raymond o>/- 
tained a signifi^nt relationship between a student's freshman performance an^ ^ 
teacher salaries.. Raymond f.urther reported that the aver4ge 'salary ^or >elemen-. 
.tary teachers appeared to have a stronger effect on student gerformance than 
•did the average salafy for secondary teachers. ^/ . " * ' « 

RibicH (1966) also conducted a study rising data from 'several sources, in- 
cluding Project TALENT. The researcher examined the relatior^hip between ex- 
, penditures per pupil and the adhievement lavelfe of approximaiMy 6,300 twelfth 
grade male students who tanked in the lowest quintile on meiXir^s of socio- 
economic status^ The effect of increases in school expenditures on test per- 
formance was greatest at 'the lower end of the expen<31ture [-ange. 'Ribich (1568, 
p. 87) observed that, "The apparent power o£ increased expenditures to impreve ^ 
.performance diminislies-^rogressively with each successive fexpenditi^e level. " 

The following yeai",- Kiesling (1969) .investigated the Relationship of schSoL 
inputs to schc\ol performance in 97 school "ddstricts in .NewTyork State. Kiesling 
utili2e4 school district records to compile the necessary school resource and / 
family background data. 7^ set of 17 independ^nt-variablejs inclUdecJ teache.r/ 
pupil ratio^ median teacher salary, ADA, and, school. property valuatipns per ^ ^ 
pupil. The dependent measure consisted o£ sixth grade ^achievement teat scores. 
The sample was divided into five subgroups based on th^ family^ "breadwinner's" 
occupation. School districts were divided into urban and* nonurban categories, . 
, The occu^^ion index was significantly related to stu(Jent achdeveitvent for all ' 
subgrou|/s^n both the urban and nonurban categories, ^iost of the tissociations | 
between achievement and per pupil expenditures in th€ urban di^triots were I 
negartive. ^er pupil expenditures did not appear to. have a significant effect, 
in the nonurban districts. - *'* \ . 

In another study Kiesling (1970) investigated the Srelationshiptof several 
school and community characteristics to student achievement in a sa^npi^ of ' • 
fifth and eighth grade pupils in 86'"school districts in' New York State. School 
data^ were obtained from the* ^asic Educational Data System '(BEDSy which was ' , 
established in New York in 1967 to collect detailed information on the state Is 
school system. The variabS^a^|^Uzed and the analyses conducted w^^ similar, 
to^te>s^ employed, in his preva?^^' study. ' Kiesling reported that the amount of 
school Resources devoted^to central administration and supervision was most con- 
sistently related to pupil achievement. In addition, the level of- teacher 
certification, especially^at the .fifth grade level, ^ind the number of sttidentp 
per classroom were alfeoyfelated positively to student achievement. 

Guthi^ie 'and 'his colleagues (1971) conducted ^ study in Michigan in which 
they. examined three re5^arch propositions invofi^ing the, re^lationsjups between 
(1) socioeconomic statusVqf pupil* and school services, (2)' schdpST services and 
pupil achievement, and (3) pupil achievement an^ jppst-sphool opportunity. These 
researchers relied upon the "Thomas Report" (Thomas, 1968) as their primary ^. - 
source of information. A random sample consisting of 52 unified school dis- ' 
tricts plus the Detroit* School District was drawn. This^ sample was supplemented 
with information collected in ^ Michigan * school districts .for .the ^EEO study in 
1966. Data from the Thomas study^ were used to make* comparisons of school > 
quality between sfhool districts. EEO»data were used to make comparisons 
between individual schools. ' • • " . 
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' The independent pleasures included our related vto school facilities, qne * 
related to instructional materials, five related to teacher charactggistics, 
^ and 'four related student characteristics. Measures of pupil performance in- 
^jlud^d pup^il test scores of ^reading ability, mathematics (inder standing, and' ^ 
verbal facility. To control for socioeconomic status (SES) , the researchers 
^ divide^ the 5,284 _ sixth, grade students intoUO sub'groups basM on occupation- 
. education index scores. They * hen calculated a rank order correlation coeffi- 
^ cient between the school input variables and student test scores within each 

SES decile.* Several, variables, inclfidihg bdilding age, library volumes per 
student, school enrollment, classrooms pe^l,000 students, teacher attitudes, 
^ and teacher verbal ability, were relateldl at^^' statistically significant level 
for a^:*' least half of the socibeconotnid^ groups,. ^d the verbal test scores. 

All of these researchers were confronted with difficult methodological, 
problems. In addition to the* problems involved in satisfying the assumptions 
underlying the ..production function itself, the complexity of the school, setting 
and the learning p^cess further complicate any research effort. First, the 
natural schooi setting itself presents formidable^obstacles. Unlike the conr 
trolled laboratory experiment, th^ school setting does not giVe researchers , an 
opportunity to carefully* control!' 'and manipulate variables to determine the rela- 
tive impact of specific inputs/; Students, for' example, rarely can be randomly 
assigned to different school treatments. Differences in classroom conditions, 
a diversity of teaching styles, teacher tumoVer, stftdent mobility, and the like 
serve to confotuid any euialysis.* % 

Second, ^the' absence of a well-dev^Io^ed theory of learning greatly com- 
plicates the research task. The specification of a production function f or* 
the 1-q^ming process— tHe relationships between the identified educational* 
inputsVand outputs — must therefore be based largely on intuition, illustrate < 
the problem^ involved, Lue'Ske and McGinn- (1975) specified several sets of causal 
relationships between and aSnong several types *of input Variables — ^aifaly, school, 
and teacl^ers. The authors used a computer simulation model to gener/te^ data 
sets characteristic of thosfe relied upon, in the previously described studies.^ 
They then examined tlfie results obtained u^ng different kinds of aggregation pro- 
cedures and regression analyses. They concluded that, because of the complexity 
of the interactions, they were \p^l^ to obtain consistently reliable es€imates 
of the predetermined causal relationships,^ i*e., those relationships built intp 
the dat^sets. Thus, they suggested that some of the "no signif^x^t effect" 
findings which i^ve been reported might be a product of the statistical tecTini^ 
.ques , employed. 

Tl^rd, most researchers have been hampered by the la*ck of disaggregated' 
data, y When 'measures oT~cent»al tendency, e.g., school or Vy stem averages, are 
used is mteasures.of input or outiSut, the true -impact of specific school re-*- 
sources is nearly always disguised.* If, for example, e3<perienced teachers 
are effective with high ability students but ineffe<j:tive with low ability * 
student§4 an analysis^as^d mean scores is likely to disclose no effect, 
liata. on in^vidual student? and teachets -withfa the same classrocJis^ 4md schools 
are badly needed. • ^ * - ' 

"Recent studies in Philadelphia and New Haven have attempted to overcome;^ 
ome- of the problemp inherent in th§ use, of aggjiagated data. Summers and Wolfe 
ri975) conducted aif in-depth analysis in ^the Philadelphia piablic school^system ^ 
using longitudinal data to. study the academic progress of approximately 2^000 
students at various grade levels i« 150 differtotVschools..* Data were relatfed to 



the achievement growth ^of individual pdpi la- betw^SIT^the end, of the th^ird find 
the sixCh grades, the s^jOJi and eiqKth grades, and the ninth and twelfth^ grades* 
Socioeconomic facto^rs specific ^^ool resources were^ caf efullyjjjed to data 
on individual pupils • ' • - ^ 

' Based on multiple regression analys|j^ Sf the data at each levei^ of- school- 
ing examined, the authors concluded 'that school inputs, such .as teachers and 
class size', and school climate variables, such as racial composition, achieve- 
ment mixture, and disruptive incidences,' .did influence student achievement* All 
types of,stpdents '(black/ white, low achievers, and high achievers) at all 
grade levels scored higher in achievement the more days they attended^'school* 
Likewise, all 1:ypes of elementary students learned more in scl^ools in which 
40 to ^0 percent of the' student body was black and in schools with a-jtarger 
percentage of high achievers. Elementary school students also* did better* in 
smaller classes and with teachers vho were graduates of higher-rated colleges.^ 
junior high school students learned more in sqhopls which were part of an 
elementary school and in schools in which there were more high achievers* These 
•students also did better with teachers who graduated from higher-rated colleges 
and with mathematics teachers who were trained in the post-^utnik, new math era 
In general, senior high schobl students displayed greater achievement in smaller 
^ schools anc^ in schools with fewer dropouts. 

In addition/ Summers anci Wolfe found that specific types of students cap 
^be helped even more if particular*types of resources are targeted "^to them* 
^Black students, • for^jkaimple, appeared to do bettfer in the smaller elementary 
, schools and in junior high schools with larger black populations* Low-achieving 
elementari^^ students did better with relativ^ely less experienced teachers, in 
smallei:' classes, and in schools with ^riore high achieyers. Low-achieving junior 
' high students did better with relatively less experienced English teachers >and 
inTscK&ols witl:; more high achievers.* High achievers, however, did better with ' 
more -experienced ^ teachers, * . .. 

Murnane (1975) conducted^An input-output analysis to investigate the impact 
^of . school resources,* especia]^ teachers, on the cognitive achievement of inner7 
''city children in New Haven, Connecticut* The sample consisted oj^75 black 
children in 15 elementary schools* Data Were available over.a\ two-y^ar period 
(Second and third graces) for one group of ^chi^dren and over .a orie-year' period ' 
(third grade) for another group. The data base divided into- three sxib- 
groups* Each subgroup was followed over the period of one school year*^ As in 
tjie PhilaBelphia study, the students in the sample were systematically matched 
with individual census blocks* ^* 

^ After examihing the effect of th^classroom as a whble on the achievement 
of children, Murnane concluded that there are important differences in the 
amount of learning that occurs in differ^ t classrooms withiA the same school 
and among different schools'. The effects of several classroom-related vari- i 
ables, such as teacher , ' peer group*, and student turnover, were carefully con-^ 
sidered* After determining that teachers exerted a critical impact on student 
achievement, MUrnane explored the trelatirfnship btetw^en' specific teacher charac- 
teristics and teacher effectiven^s in math and reading instruction with certain 
types" of pupils. . , . ^ - " 

Based on tj^^^esults gained by regression ai^lysis techniques, Murnane re- 
ported that l^tfckground factors and previous expediences. had a' greater influence 
upon- student Sheading achiepment than upon mith achievement* Differences in ^ 
the quality of cl^ssfoom environments were found to exert a 'greater effect on ^ 
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Student math achievement than on reading achievements Murnane also found t!Rat ' 
black feathers with less tharf six years of experience were more effective in 
teaching reading to black children than were white teachers with. similar . 
teaching experience • At the same *tiroe, a high rate of student turn9ver in a . 
class was found to have an adverse effect oit children's reading' achievement, 

particularly on the progress of high achj^vers^ , ' ' ' , ^ 

By.usin^ disaggregated data, both x>f these studies 'reveale^ imports^nt 
findings. Many school resources affect different typ6s of students in dif- 
ferent ways? few school resburces consistently effect all students <equally* 
Clearly, an important aspect of the^ educational process is the unique inter- ^. 

^ action that takes place between certain types of school resources and certain 
types of students. Udw achieving students, ^ffcr example, appeat to learn more* 
with relatively inexperienced teachers while high achieving students seem to 
learrymore with^experienced teachers-: Small classes apparently hftlp low 

^ achievers but are not ^particularly important for average or ^igh achievers. 
Only research which fOcuse^ on individual students within specific instruc- 
tional settings can offer insights into these extremely complex interactive 
relationships.*' • , , 
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PrANNING-PROGHftMWING-BUDGETING SYStEMS 



^ As discussed previously, the systems .approach" has heen used in the quest 
,for efficiency and productivity in educational organizations. A well-conceived 
pianning-programming-budgeting system (PPBS) itiay b^e viewed as an operational ap- 
plication of systems analysis. In general, a^ PPBS would include as a minimum 
the following procedures: (1) specifying the objectives, ,(2) identifying alter- 
native programs to achieve the. specified objectives, (3) estimating as accurately 
as possible the costs and benefits associated with each (5f the alternative 
courses of action, (4) selecting one or more programs to implement/ \( 5) moni- 
toring the implemented programs and evaluating progress toward attainment of 
specified, objectives^ and (6) feeding back the results of the evaluation tp 
modify t:he program ^and inc^eas^ its ?ef ficiency.' . ^ ' 

PPBS stresses strategic planning and the role of analysis in. the budgeting 
process. Fisher (1971a), one of the first writers on PP3S, sumnjarized the, most 

salient characteristics of PPBS under three basic headings: x 

of 



1. fjThe^ structural 'aspects <?# program budgeting ^r^ concerned with 

establishing a set of categories 'oriented primarily .toward 
"end producV* or. "efnd objei^ve" activities that are meaning- 
ful from a long-range planjjfing point of view. In such a con- 
text, emphasis is placed on provision for an extended time 
horizon — spme five or even ten or mofe years into the future. 

2. Analytical pyocess considerations pertain to various study, 
activi'ties conducted as an integral part of 'tHe^ program- 
budgeting process. The primary objective of this type of aria- 

. " lytical effort is to systematically examine alternative 

courses of action in terms of utility and cost, with a view \ 
to clarifying the relevant choiceg (and their implications) 
open to' the;decision makers in a <:ertain problem area. ' ' \ 

3. Information system considerations are aimed alvsuppprt ^ the* 
• f iri^t two itemsv There are se<reral senses in whicR this is 

important the primary ones being (1) pr<^gress reporting and * 
cpntrol and (2) providing data and^ information to ^rve as- * 
a' basis for the analytical process— -especially to facilitate 
^ the development of estimating relationships that will permit 
^ taking estimates of benef it^ and costs of alternative future 
courses of -action. . . 
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f 'while eipphasizing the planning function and the , systematic evaluation 
of alternative programs., PPBS also , accommodates the-qther major budgeting 
functions of financial and managerial control. Budgeting systems at federal, 
state,' and local, governmental levels have always consisted of these three basic 
functions but each has been emphasized to a different degree throughout phe 
last 50 years \ V ' " 

Schick (1971) describeja\hree successive stages of budgetary reform at 
the federal level in the United States^ The first stage began around 1920 with 
the Budget and Accounting Act of -1921 • . Thrpughout the 1^20s the budget .was 
viewed basically as, an instrument for controlling organizational spending and 
cutbin^ administrative abuses. Changes in budgeting practices therefore were * 
primarily concerned \fith development of a reliable system of expenditure con- 
trol. Characterized by an input orientation/ the typical budgeting. structure 
during this period systematically" ^detailed the "objects" of expenditures. 
This object-of-expQndi€ure glassif ication scheme, generally called a "line- 
item" budfget, simplV provided a listing of the expense o]5>jects, such as salaries, 
supplies, and equipment. " ^ ' 

The second stage . of 'budget reform, according to Schnick, extended from 
the mid- 1930s to about i960 and was associated with ^ the rapid expansion of 
governmental activity during that period. The ^scientific management approach 
'to adminisferation strongly influenced budgeting practices throughout the 
1940s. Reflecting this management or ientatf on > ^udgetin^ sy.stems were used 
increasingly to assess and improve administrative* performance and worker out- 
put. During, thisperiod performance standardsr and work measurement tec^iques 
were introduced Snd developed. In L949, the first Hoot^^r Commission advanced 
the concept of performance budgeting by recommending that expenditure classi- 
fications be based upon functions, programs, and activities. (In compafcrison, 
the management-oriented functional-object budget delineates 'broad categories 
such as administration, salaries; plant operation, plant maintenance, fij^ed 
charges, cafJitJal outlay, and debt service.) , ' . 

The third stage of budget reform began in >the early 1960s with the intro- 
^duction of PPBS. The adppt^^ ojp^PPBS by the Department of Defense (DOD)^ in 
1961 was preceded by a considerabO^e amount of research by the Rand Corporation 
^thijoughout the 1950s (McKean, 1958; -Hitch & McKean, \1960) . Basically concerned 
with analyzing the performance of military weapons systems for the POD, several 
economists associated with the Rand CorporiMrion developed the' basic fconcepts 
and techniques of PPBS. David Novick (1964)*/ a Rand economist, edited, a 
series of publications which became the first definitive work on program 
budgeting. « 

In 1965, President Johnson directed the major civilian agencies of the 
federal government to implement PPB. systems similar to the one ^sed in the 
DOD. InAuenced by federal budgietirtg reforms, various state and -local govern- ' 
ments also initiated. projects to develop and implement program budgeting 
techniques! In 1968, at a- time when some schbol distiricts' were beginning^ to 
•experiment seriously with PPB systems. Hartley (1968) explored the application 
of PPBS cohcepts t6 e<iuGatibn and Berfson (1968) advocated the .use of PPBS in. 
education^ to reduce deficiencies in 'the allocation. of public resources.' A 
short time later,, the Rand Corporation^ published a comprehensive 'report ex- 
plorir>g th6 potential use of 'PP^S. concepts and techniques in educational 
planning (Haggart^. et al., 1969). ' I ' ' \' 

^ The performance budgeting aipproach advocated in the 195t}s is based uppn 
concepts drawp firom tljfe scientific manacfement movement. It concentrates on 
inputs and uses cost accounting procedures, .In cofitrast, program budgeting « 

. 1 ,• .' . .■ ■'. • -27. . •.. • 
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is 'i^ased upoo concepts borrowed from systems analysis and economics. It 
focuses on outputs and utilizes program accounting techniques • More impor- 
tantly, while performance budgeting a^ssumes that objectives are fixed^ pro- 
gram budgeting assumes that objectives may be variable. In a PPBS> then, 
systemat;ic analysis of relevant alternatives ' can lead to a revised statement 
of objectives. The program budget becomes^ in effect^ a policy statement. 

A fully ""operational PPfeS includes multiyear program and -financial plans , 
for projecting future cost implication^. To' -estijnate costs and benefits 
over an extended time perit>^a^ PPBS requires an end'-product claisif ication 
system that systematically relates expenditures to designated objectives. 
Accordingly, an appropriate program structure must be devised t6 facilitate 
grouping^-of the activities and their associated costs into appropriate cate- 
gories "based ^ upon the'ir relationship to objectives\ Program structures 
typically are hierarchical classification' schemes that group an organization 's' 
activities into programs, subprograms, and progrl^ elements. ♦* A program may be 
define4 as a set of acti^vities that function together to achieve the saine . 
objective's) . ^ . \ . , 

Grouping of budgetary data by programs facilitates the analytical EPro- 
cess. While there is no single ideal" program structure, classification 
schemes should possess certain characteristics. For example, progreun struc- 
tures should be multidimensional, i.e.,f. they should provide for the categoriza- 
tion of activities^y several criteria. Various Cross-cutting sets of program 
categories may be needed for different; analytical purposes. In addition, ^ 
classification schemes should be flexible. Program categories should be^ 
developed and altered as analytical needs^ qhange. • 

A number of educational program formats are possible* In the most 
conunon approach, the f>rogram cost structure is simply based on grade levels. 
Programs might include the, kinder gar ten thi^ugh third grades, fgurth throug] 
jsixth grades, seventh and ei'^hth^ grades, and the Kigh school years. In a 
^second approach, programs are based on curricular. areas and costs are appor- 
tioned to specific curricular areas, such a^* reading, language^^J^ts, mathe- 
•matics, science, and social' studies* ' In a third option, a combination of the, 
first twQ approaches, program costs are organized by siibject matter , and by' 
grade levels,. Th^development .of this kind of progr^-accounting" scheme, 
however, requires access to a sophisticated^system* for information storage 
and' retrieval^. . " ^» * 

Current school budgeting practices do not reflect widespread acceptamce 
of program budgeting techniques. Most*" school budgets still employ traditional 
classification schemes which emphasize the fiscal control or accountability 
ksp^t of the budgeting process. The following major expenditure 4ccounta^ 
i;ecoiranended by the U.S. Office'of Education in/^f^, are still usedXextgrj- 
sively by school districts today (Reason &^W)ate, 1957): ••^-^ 

Expfenditiire Accounts , 

, Administration ^ 

V- Instruction 

• -? 

<^ . Attendance and health services 

Pupil transportation' services » ^ * 

Plant operation* . ' ' ' 

Plant maintenance 
Fixed charges^ 
^ ,^ Food services and student body activiti'fes 





Community services , - 

Capital outlay 
• Del?t service from current funds 
>^ Outgoing ^transfer accounts 

lile this accounting .format is useful for fiscal control purposes, tK&e 
b»oad functional categories do not provide the detailed copt finf ormation 
nec'essary to improve fresource allocation decisions. *Por ^example,' the largest 
bu4gat category, instruction, typically includes the following subcategories.: 

" ^ . Salaries 
u . ""^ Principals 
• X ** Consultants or supervisors. 

* ' '^^ \ Teachers 
^ ^ Other instructional staff , * - 

^ Secretarial and clerical assistants 

: Othe^ salaries fo;p instruction <• 

Textbooks " , 

School libraries and audiovisual 
Teaching supplies 

Other expenses \ ' 

- . ^ . ^ r * ^' 

Clearly, improvements in school budgeting cind accounting prpcedures aVe 
necessary to facil;Ltate systematic analysis of resource allocation decisipns. 
Guthrie (1973) sbg^ested the following format as a more useful approach to 
budgeting and accoUntin"^ for the instructional function. 

^ • * . Iristructiofl-r^Reading 

/ • ' > 
' ^ Developmental reading,* totaf'l 

Elementary sphool services , . . - * 

Pergonal services ' 
* Supplies ^ \^ 

Capital outlay , * . ' 

Junior;high services ^ ' . • 

' Personal Services ' . « . 

Supplies 

Capital outlay ^ 
Senior high services * , » 

Personal services ' . 

^ ' Supplies 

Capital outlay 
Remedial reading-, total,' et^ 

The above format categorizes expenditures by level of., school progran^nd by 
type *gf instn|<ftion offered. By utilizing this type o£ cost struo^xe, alloca- 
tiop of fiscal resources is shown in considerably greater detail. We emphasize 
the importance of preparing such a functionally detailed budget at the indivi- 

, ^ual school level, sihce" district-wide figures can' obscujfe the impact of school 
r^so^urces at the school, classrooinv. and student levels. With this type^of cost 
* information, however, educational deci^iph makers and school administrators are 

»>j.n a much better position .to assess the cost' and effectiveness of specific in- 
sJtructional ppogrcims. - ■ »^ 
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As IS often true with innovations in educatioSi, claims concerning- the 
benefits of .PPBS were exaggerated and it has fallen from favor in recent ^ 
years., Conv^ting a traditional functional-object budget intdN^ mul^tiyear 
'document stressing long-range planning and focusincj on operational oijjectives 
is a difficult task. PPBS, for instance, rests on t\ie assXimption that objec- 
tives can be specified. Yet the ^specif ication of TDbjectives invariably in- 
volves value judgments. No PPBS can, resolve the inevitable disagreen^ents 
which arise concerning the objectivjes which should* be pursued and the 
priorities which should' be assigned^^ Nevertheless, PPBS forces decision 
makers .to clarify their values by^ identifying and pperationalizing' or.ganiza- 
tiorfal Objectives. Despite its shorircomings, then, PPBS has considerable 
utility as a management tool 'and clearly sets i:he sta^e for cost-ef fectivfeness 
analyses. ' * \ .< % 
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VI. 

COST-EFFECTIVENESS 



Analysis 



Systems analysis, ^economic theory, PPB? aljl emphasize one basic 
notiori~-^.the examination of alternative means to achieve given ^hds. Co^t- * 
effectiveness analysis implies that the, preferred alternative (s).^ will be. 
selected on the basis ^ of efficiency criteria. Characteristic of most cost- . 
effectiveness definitions is that of Seiler (1969, p.a) who defined cost- 
effectiveness ctnalysis •as'^^a 

procedure by which the cost's of -alternative means of achieving ^ 

a stated effectiveness, or, conversely, the effectiveness of 

alternative means for a given cost, are compared in a series of 

mimerical indices. The objective of the analysis is to isolate 

the alternative or combination of alternatives that either gives - 

the greatest. expected effectiveness* for a given expected cost or 

a given expected effectiveness fbr the leas£ expected^ cost. \ 

^ Several writers have advocated this >asic approach for comparing alter- 
natives ii\ cost-effectiveness analysis.1 Haggart U^69, pp. 152-159), for 
example,, in addressing educational planning, foiind it useful tb hold either 
the cost (i.e.., budget level) , or effectiveness (i.e*. , student achievement) 
constant. In other words, meaningful comparisons can be made between tfie cost 
of different alternatives for achieving a .prescribed effectiveness level or 
between the effectiveness of .different alternatives for a fixed budget level. 
There is no sound basis for comparisons,.. however, if the ^alternatives differ 
In terms of both cost and effectiveness. Haggart argues, tha't maximizing the 
cost-effectiveness ratio for the sake of the ratio alone can lead to ridiculoxis 
^ytremesr-J.ik« zero to infinite cost or zero to infinite effectiveness^ 

To Jnaximize pupil performance, educational decision makers are concerned 
with using school ^resources as effectively as possible. Cqst-e/fectiveness 
analysis is a tool that can assist educational decision makers irv. choosing 
among alternative courses of action as they seek to allocate resources effi- 
_^ciently.-^^In their discussion, of educational, planning,. Carpenter and Haggart 
(1970, p. 2) indicated that cost-ef f ectiveness *analj»i's may be us^ (l)r. to . 
help assess the relative vbrth of several innovative programs with the same 
educational outcome (such as improvement in readin'g achievement), (2) to 
determine whether a single program is becoming more or Jess effective as 
time passes so that step?^ may be taken to improve it, if necessary, or* 
(3) "to help assess 'the relative^ worth of , the same program for different 
student populations (such as those with different socioeconomic backgrounds) 
or in different school settings. ^ , ^ 
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Researchers at the Rahd Corporation were among the first to examine th^ 
role of analysis in the program budgeting context/ Fisher and Quade agreed 
that most major long-range planning deciiSion problems must ultimately be re- . 
ftolved primarily on the basis of intuition and judgment and that the main role 
of cinalysis should be to try' to sharpen this intuition and judgment. After 
emphasizing that there exist no universally accepted procedures or rules for 
conducting cost-effectiveness 'analyses, each researcher described some basic 
characteristics of a sound analysis. ' ' v . 

Fisher applied the term "cost-utili'ty" to the analytical process involved 
in program budgeting. For Fisher, the process of analysis included the syste- 
matic examination of relevant alternatives, the design of new alternatives, 
and the modification of initially specified objectives. Fisher (1971A) item- 
ized the major- attributes of • cc5st-utility atnalysis asffollow^s: 

1. The ^sential characteristic is the systematic examination 
and comparison of alternative qourses of action. .The courses » 
of action are those whi^h might be taken to achieve specified 
objectives in some future time period. 

2. Not only does one consider th9se relevant altei^natives that 
are revealed under initial search but 6ne constantly se'eks to . 

> design new alternatives. * • , *^ ^ 

' ' ' * V/^ ' • . 

3. ^ The systematic Examination of alternatives, both old and new, 

frequently suggests some modification in the objective? thentr 
selves. This is to be encouraged.^ . ' 

f 4. Critical appraisal of alternatives rests mainly on the ass^ss- 
* ment of costs, both direct and indirect and in the. future as 

well as in the present, and the asses^sment of gains or bene- 
fits that accrue to each'^of the different coiirses of action. 

5. While most of the work is quantitative in nature, it should 
^ frequently be supplemented by qualitative analysis. 

6. The time periods of thq analysis is an extended one. This 
creates prObleijis ir> the treatment of^ iincertainty that should 

' be 'dealt 'with -explicitly. • 

, Similarly,^ Quade -(,1971, p. 295) contended that* the Aise of the label "cos t- 
ef fectiVeness analysis" placed undue emphasis on a single aspect of the decision- 
making process. " He argued that*- other aspects of a problem are more important 
th^n the comparison of alternatives — "The specification of sensible objectives, 
the determination of a satisfactory ,way to measure performance,* the influence ' 
of c3nsiderations that cannot be quantified, or the design of better alterna- 
tives.*' Quade's 'principles of sound analysis specified that (1971, pp. 299- 
300) : ' . _ ' ' ^ / 

1.^ The' right problems must be. tackled, i*e., discovering the ap- 
^ propriate objectives,, searching out good criteria for choice,^ 
and choosing the best set of alternatives to compare.. V- 
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2. The analysis must be systems oriented, i.e., finding the 
interdependencies^that exist in a system's parts and study- 
ing the entire complex system e^erf if it requires intuitive 
judgment. 

2. Uncertainties should 'be recogniied^ari^ an attempt ^should be , 
mad^ to t^ke them into account^^iJS^evaluating their impact on 
the answers. .Answers change in res6onse to changes in assump- i 
^ tions and/ estimates. " ^V. 

4. fhe ^alysis should attempt' to discover new al^ternatives as 
well as to improve the obvious ones. 

5v The analysis should strive to attain the standards traditional 
to science even though in problems^ of public policy the scien-^' 
tific method of controlled repeated experiment cannot used. 
These are (a) intersubjectivity : results , obtained by processes 
that can be duplicated by others to attain the same results; 
(b) Gxplicitness: use of calculations, assumptions, data, 
and .judgments that^are subject to checking, criticism, and 
disagreement; andMc) objectivity: conclusions do not depend " 
on personalities, reputatipns, or vested interest; where 
*' possible these conclusion^ should be in quantitative and ex- 
perimental terrps. « ^ ^ 

Those who attempt to apply systems analysis and co'stref f ectivehess anal- 
ysis to problems must confront major elements 6t uncertaint^y because of the 
ex^iended time horizon, which usually involves a projection into the 'future of 
at least five to ten years. Fisher (1971a, pp. 190-91) advocated the use of 
sensitivity analysis, contingency arialysis, and a fortiori analysis in 
treating the troublesome feature gf uncertainty inherent in most long-range 
plannin^^s problems. 

*In the first technique, sensitivity analysis, the expected values of 
the* key parameters are varied.^ If the costs of particular resources are 
uncertain, an array of high; medium, and low values can be used to examine' 
the various alternatives. In this manner, an effort is made to determine 
how sensitive the results (the ranking of the alternatives being considered) 
are to changes ip key parameters. In the contingency analysis 'approach, 
the effect on the ranking of the alternatives of changes' has been called 
the "what if" approach. In it, a major change in ^the general environment 
can be assumed. ^ In the third technique, a fortiori ^analysis, all questions . 
of uncertainty may be resolved adversely with regard to the intyiitively pre- 
ferred ai^Temative. The preferred alternative is then xrompared with^ the " 
other p6ssible alternatives. A. strong case for the preferred alternative 
is assumed if it still compar^sS ^favgrably with the other alternatives 
despite the built-in adverse^^ofdi tions. 

While the problem *of tifeating uncertainties presents some major diffi- 
ctJlties,'* It is not the i>Tlly problem- which^ must be confronted. Analysis o' 
t^e cost^ and ef fecti<:feness of a range of alterl\at^^7es also involves prob- 
liems. In the following section we will examine the basic notion of economic 
costs, various types of costs, and appropriate cost comparisons. Similarly, 
in the sefction on the analysis of effectiveness ^e will consider the problems 
inherent in measuring program outcomes. ^ m 
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ANALYSIS OF COSTS 



A discussion of costs must begin with the^notion of "opportunity costs #" 
which can be defined in terms of "what is given up" rather than "what is put 
in (Bowman, 1966]." The concept of opportunity cos ts^ involves the p;roblem of 
choice and the examination of alternative' uses of resources. If resources 
are consumed in achieving one objective, they cannot beVused to accomplish, 
'other purposes,. The real cost of any alternative, therefore, is the sacrifice 
incurred because -the decision maker chose not to pursue/ some. other alterna-- 
tive. "Costs occur when a choice is made among several desired benefits. In 
short, costs are benefits — benefits given up by choosing to do one thing 
rather than another. [Hal ler, 1974]." ^ ' / 

In amy analysis, this broad rlotit^y^^f costs Jfiust be considered along 
with the more obvious direct expenditure it^msA' Schultz <1963) developed the 
concept of\ opportunity cost within an educational context by examining the 
costs of a| college education to illu|itrate^ the qigrvificahce of foregone alter-' 
natives, ywhen people withdraw from the labor force/ to attend school, society 
Joregoej^ their productive services and th^ indivirf^ls themselves forego 
eaa^ftifigs. The costs of these lost' earnings to both society and individuals 
must be calculated as part of the total inves.tmeht in a colle'^e education. 
Schultz suggested that approximately 60 percent of the cost of college coh- 
,sists of foregone earnings. ^ * - 

Similarly, Thomas (1971, p. 32) argued that time sj^ent within* school, 
might better be governed by the principle of "foregone learning." He explained 
that "the implication here is that the cost of a given curriculum or of a given 
instructional procedure is measured in part by foregone oppo?:tunities to devote 
students' and teachers* tim^ to other curric|^la and procedures." The time -n 
students spend study halls, for example, cann^'be used for, classroom in- 
struction in English, mathematics, or other subjects. Likewise, when teachers 
ar^ a'ssigned to supervise a lunchroom or playground, the school foregoes^ their 
presence in classrooms. Administrative* decisions involvi,ng the scheduling of 
pupils and teachers are not without cost considerations; the time of pupils 
cmd teachers, as well -as school space, equipment, and supplies, are all limited. 

Several writers have addressed , the 'pitfalls 'involved in conducting cost 
analyses. Bickner (1971) thoroughly delineated the problem^ in performing 
cost. analyses )of military programs. HaLler (1974) discussed many of the same 
problems within an educational context* Typically, two basic. prodedures are 
followed to dfetermin^ the costs of program alternatives. First, a* listing of 
the specif icy resources required for each program is developed" and then a 
monetar^s^v^ue is assigned to the resources identified. i 

After all necessary resources have been identified, a comprehensive, input 
structure that accurately allocates specific costs to particular programs is 
developed. As discussed previously, a classification scheme should be designed 
to facilitate the analytical process and permit ^anlngf^ comparisons. The 
major expenditure categories for personnel, facilities,' equipment* and 
materials should be divided into subcategories to delineate costs in ^s mucfi 
detail as necjessary. At^the scime time,^/the classif itation scheme should also 
treat the time of students and school personnel as a valuable resource. i 

The input structure also shoi^ld relate expenditures to their purposes in 
the program. Tisher (1971b), for example, discussed the notion of a prograjn 
TiXfe cycle. New educational programs, as well as changes in ejcisting programs, 
normally involve a stream of costs ove* an e:^feended time period. These costs ^ 
Cctn be segregated into the following three categories (Haller, 197.4, p. 420): 
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Research and development costs — resources required tso develop ^ 
the program to the stage where it can be introduced into -the 
'system. For example, the time a science* coordinator spends in 
investigating a^ possible program, money used to hirfe consOl- 
tantfr>» and evaluation 'effort are appropriately classified as 
•research and development (R & D) costs. 

Investment yosts — costs necessary to implement the program. 
Equipment purchases and the costs of running a Workshop to » 
train s^aff ^re examples of investment costs. 

Operating costs — recurrir^g costs required to operate the program 
over time. Maintenance of equipment^ salaries eft personnel, and 
cost'of supplies are examples. * f * 



- ' An example of these costs over the lite cycle of a program is illustrated 
in Figure 1., Each type of cost typically behaves somewhat differently over 
time and is incurred at different stages in the implementation. process. 
Researcn and development and investment cq|^s usually have their, greatest 
impact early as compared with operating costs iwhich occur later but extend 
over a potentially much longer time period, in examination of R & D, in- ^ 
vesTment, and operating costs helps focus attention on the time horizon of 
th% program and the projection of future costs. ' . 



C^STS^ 



Operating c©sts_ 




R&D 
sCOStS 



Figare 1. Tbe relaticnsHlip between 'costs and the li^ cycle o'f a program. 
- Source: Haller, 1974,"^ p. 421. / . ' 

Decisions made today are likely to incur costs over 'a period of several 
years and may limit possible options in the future- Therefore, cost-effectiveness 
studios must >e congerned with th^ .estimation and analysis of future costs'. Costs 
can nejpr be estimated witF ^solute accuracy and the further into the f utyre ' 
one attempts to project costs, the more uncertain the analysis. While the input 
structure shoul^ accommodate a .comparison of alternatives over an ex£*ei!ded 
period, it generally is not advisable to develop a-* highly detailed set of"^st 
categories when a relativgly lon^g time -period is ihVolved: The cost of -construct- 
ing and operating such a cost structure would prqbably outweigh the benefits o^>- 
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tained, i^e,, would npt be costVef fectiveL As mentioned previously, however, ^ • 
techniques such as sensitivity analysis and contingency analysis may be used 
to sharpen the- evaluation of^ alternatives ♦ A ' , ^ ' 

Once?a multidimensional input structureMas been developed; a good 
general rule i^ to allocate time to the analysis of eacK cost category, in a^ 
manner roughly proportionate to each category's ^Jiare of the total budget* 
.Thtis, the personnel category, which typically represents 70 to 80 percerit of 
the educational budget ,4phou Id receive closest scrutiny. Cost6^ f or personnel 
salaries Snd wages generally can be assignee^ in a rather straightforward manner. 
Direct salary/payments, howler, do not reflect fringe benefits and other costs'* 
that employers must pay^ Payments for item$ sucK as retirement, social security^ 
health insurance, and life insurance, frequently comprise 20 to '30 percent of a 
school district's personnel costs. In addition, the value of* volunteer icontri- 
biitions — such as library aides or playground supervisors — -should be included in 
the analysis. , . , ^ ' . * , 

A school district's salary schedule. can be Used for projecting future 
cosits of educational programs. Teacher salaries "are determined largely by two 
variables, years of teaching experience, and level of education. Thus the com- 
position of the staff in terms of experience and training wil'l have considerable 
impact on the cost of a program. A staff cqmpos^d primarily of befginning teachers 
will cost substantially less than a staff composed primarily of experiqnq^d, 
highly trained tefeichers. Information useful projecting, salary costs can be 
obtained by examining a district's sala;ry schec^ule along with the characteristics 
^f the teachers employed. Future costs Ccin th^n^be projected by estimating the"^ 
movement of staff through the * salary schedule^ over a program's time horizon, as 
well as projecting the impact or Jjoss^ble changes in the schedule itself. 

Costs of facilities or space, als^re^esen.t ar major ^rexpenditure category. 
These costs ara generally more dif fi^ult^ tg estimate than the salary schedule. 
Thomas (1971, p. 44) identified five components of the cost of, classroom space* * 
as (1) interest on unpaid debts; (2) foregone, interest 'on equity; (3) depre- 
ciation, or annual decrease in value; (4) overhead, or heat, light, and power; 
and (5) maintenance. ^ * * ' * * ' 

To^ illustrate these costsy Tho^tias used as an examjile a 1,200 '^square foot 
biology laboratory. He, assumed that another 25 gercent of that space (300 
square feet) is needed for supporting space (corfidors, ^tc), that the cost 
of construction was $16 per square foot (making a total cpst.of $^24,000), arid 
that^thg present value (after depreciation) is^$20,000, of which $^.5,000 is 
still owed. The cost of ^ the space is calculated as follows^ ^ . 

Interes,t on debentures (aissume' 4 percent pf $15,000). * ^ $ 600 ^ 

2\ Imputed interest pn equity (assume 5 percent of $5,000). 250 

/ ? . ^ * 

Depreciation (assxime additional expected 'life'*^ '20 years).. 1/^00 

4. . Maintenance. ' 400 



5 . Overhead ( 1 ight , , power , hea t ) 

TOTAL 'Cost » . ^ - * 



400 



$2,650 



/ Space requirements are aiv iinj^tant ingredient ih any cost^ arialysis. Space 
is a limited resource and deci^Qns regarding its Use involve opj^ortunity costs. 
For example, classrooms used ror Vtpdy halls c2mnoixbe used simultaneously for 
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dfejier purposes/ When considering; the value of foregone alternatives, the 
cost f or spac e >may constitute a significant portion of total co3ts. Since 

-new educatixmal programs seldom requii^ construction, of new facilities, how- 
ever, cost; of space is often disregarded, , ^ , ' 

In- comp.ar^ison to the opportunity .co^ts of time or expenditures for 
salaries and facilities, the cost -of equipment' and supplies generally- consti- 

^tutes a small part of a total program budget, --Educational programs particu- 
larly are labor-ijvtensive while equipment and supply costs are generally minor 
considerations. Frequently rule-of-thumb estimates for equipment and supplies 
will suffice. ^^S^or example, the total supply cost can be estimated by dividing 

, total expenditures for supplies 'by the total nxunber of pupils enrolled and 

'then applying this per pupil cos^ to the t^^er of pupils served by the pro-* 
gram under cdnsiderationv * . • ' • • 

Bickner (1971, p. ^35) pointed out that the major responsibility of 
the\cost analyst is to distinguish between relevant and- irrelevant costs: 

L-^All costs are ^relevant to some decision or other, past or 

future, for otherwise they would not be costs. The responsi- 
bility of the cost analyst, however, is hot simply to add up 
axxy and all tosts indiscriminately, but rather , , , ta i-den- 
tify and measure that particular coUection'of costs- that are 
^ contingent upon a specific decision or cHbice under considera- 
tion. • . . Any» cost that will be inqur'red, no* matter what 
choice we make, any cost that must, be borne regardless of the * ^ 
decisions at hand, is not a cost of that particular' choice or " ' 
4 decision, 

••c , * - 

Costs can be categorized in a number' of ways for analytical purposes. 
Economists, cost accountants^ and cost analysts apply a variety of terms to 
distinguish one type of cost from another, "for example, sunk ahd incremental 
costs-, fixed and variable^costs, and recurring a«as nonrecurring eosts! These 
different; categorizations of costs can be helpful ih distinguishing between ^ 
relevant and irrelevant costs. 

- Since costs are coj:isequences of decisions, one may contend that; the rele- 
, vant costs are those which lie in the future! The cost of constructing a . 
'building, for example, lies in the past. These sunk costs are the result of 
past choices. They are therefore irrelevant to current decisions concerning 
how to best util^e space in the future. On the other 'hand, ^decisions regard- 
ing tne future use of a building involve incremental costs. The additional 
costs that wilJ^ -incurred^ because cTf some proposed change fepr^ent meaning- 
ful alternatives. Only incremental costs are important in this sense; sunk 
costs should not be included in the analysis since they-only serve tb confuse 
the decision maker. • * • 

A distinction shou^ also be- drawn between fixed and variable costs, 
l^ixed .costs, often called overhead costs, typically include administrative 
salaries, maintenance, depreciation, and the like. -They generally -remain 
stable and do- not change j*ith expansion or contraction of a specific school 
program, for example, a decrease or increase in en»<Jllm€nt in a science 
program does not usually require a change in administrative staffing. Variable 
•costs, however, are closely related to the size arid output of a program, e 6" 
the number of textbopks required varies directly with changes in enrollment. 
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When yfestimating the costs of expanding a program, fixed costs which do not 
aff^t the decision at hand sho\^kbe excluded from the analysis. 

Another useful^ distinction can be made between recurring and nonrecurring 
costs, As l:he name Jlmplies, nonrecurring costs are incurred only once during 
a specif ied' time period. An example of nonrecurring cos^s would vbe e?cpendi- 
tures for* an inservice program to train teachers td use new curficular pro- 
ducts. Recurring costs are incurred repeatedly (usually yearly) and 'include • 
expenditures for salaries,, materials "and supplies, a^d physical plant bpera- 
tion. As in the previous 6x^ples, only those -costs incurred within the 
tim^ frame of the decision should be included in the analysis. ' ^ 

The distinctions between sunk and incremental, fixed and variable, and 
recurring and nonrecurring costs*^ are not absolute. Bickner (1971^ pp. 36-37) 
notes that cost's are related to the time, scope, and horizon of the decision 
ijnder analysis: * 

, That is to say, the costs of continuing a^tertain program (and 
'—the dividing line between sunk and incremental costs) depend ' 
upon the precise time of the decision. The costs of Expanding 
- a progr^ (and ,the dividing line between fi'xed and varilable 
' costs) depend upon the initial arid the revised' scope of the 

/^^roqram. The costs of extending a program (and the dividing _ 
line. between recurring and* nonrecurring costs) depend upon J:he 
initial and the revised termination dates, or horizon / of tKe 
progrcim. ' ^ • 

, 9 * 

Once the relevant costs have been identified and estimated, ithe costs 
pf alternative courses of action -can be compared. The most common analytipal 
procedure involves a comparison of ' alternatives/Jin terms of their total, 
average, and/or marginal cbsts. The most appropriate cost comparison -depends 
on the purpose of the evaluation being, conducted or the decision under consi- 
deration. / " 1 , 

A ccOTiparison of tota/ costs is .usually necessary when a decision involves 
the adoption of, a new prtfgram. The decision maker will"" need 4ata regarding 
the cQst of the Dresent trogram' as well as the incremental costs required 
^ implement anyVodificltions. In deciding to retain, replace, or modify a 
program, the totai)^cost tmplicationsTof any ch6i*ce must be projected over a 
specified time period. A comparison pf total costs can be quite useful^ in 
co$t-effectiveness stuGies which deal vith alternatives of about equal effec- " 
t^iveness. Assume, foJ^example, that two^different reading programs produce 
the same level of student achievement. The analyst could simply calculate the 
cost of each alternative and recommend that^hoice yrhi oh minimizes total ex- 
penditures • - ^ ^ ' V * 

The compilation of average costs , hoyever, might be more appropriate 

if two reading programs varied considirablj. Assume one program reli^ J 

heavily upon computer-assistedi instf.i/ctijLon ^d otheif technological aids"^ 
while the other utilized small group instruction and required additional ^ 
teachers. Average costs, of course, can^be estimated for ^ny number of dif- \ 
'^ferent units— ^chools, classrooms, teachers, students, and the like. EJcamples 
range from yearly per pupil expenditure to cost.per st'udent hour of instruc-' 
tion. The cost unit employed will depend upon the particula^ decijsion to be. 
made. In general, the unit of cost .used should be closelifeHreiated to the 
major cost variables involved. ]for exaumple, bhe^osts of supplies will 
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vary "directly' with the number of students served by a program, where^as equip- 
ment costs are likely to be a function of the number of classrooms utilized 
by a program, . - ^ 

A discussion of average costs leads directly to "the notion of marginal 
costs. Marginal cost may be defined ajt-jfehe incremental cost of providing , 
one additional uni^ of a .specif ic good' pr service. The marginal cost approach 
IS particularly helpful in analyzing the impact o^ a decision concerning ex- 
pansion 'or reduction of an existing program. Marginal cost comparisons can" 
^e used to help dete^ine the important distinction? between fixed costs and 
variable costs discussed earlier. The costs for teachers and space, for 
examj^le, may shift from fijced to variable and back again, depending upon 
variations in studeat enrollment. ^ Thomas (1971, pp. 45-50) explored the 
<r^onomies of scale in a typical .high school program by analyzing the cost of 
adding another biology class. He assumed that a single biology lalp can serve 
a minimum of 120 students (24^ studeats' per period for a five period day) 
and a maximum of 200 students if overloading is permitted. Fiqurp 2 illus- 
^trates the behavior of the cost per hour of instruction *as enrollment increases- 



0 so 100 150 200 2S0 300 35T) 400 450 500 550 600 
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Figure ^. Average cost per student per hour of instruction in biology- 
Source: Thomas, 1971, p. 48. 



In Figure 2, the average cost, per studer^;*^ Ijour declines rapidly as 
student enrollment approaches the maximum capacity of the 'biology* class. The 
average cost line, however, changes direction once classroom* capacity is 
achieved. For purposes of the example, it was assumed that the- enrollment of 
the 20lst student requires an addftlonal biology 1^ and ^eacher. ^This hypo- 
thetical ex^ple' illustrates that the expansion of a school program involves 
relatively small marginal costs per additional student until a capacity point 
is reached. At- that point, the marginal cost increases dramatically. While • 
seemingly a simple task, then, ,it should be apparent that identification, 
measuren^nt, and^ comparison of costs involves numerous potential pitfalls. 



39 



ERIC 



V 



ERIC 



34 



AfiALYStS OF EFFECTIVENESS 

• « « • 

Cost^^re used to measure resources that go into a program; the concept 
of ef fec}:iveness is .used to assess the oi^tput of a program. Program effec- 
tiveness can be defined in terms of goal attainment-.^ In short?, how much pto- * 
gress has been =made toward the accomplishment of designated object^ives? 
'Accordingly, an analysis of the effectiveness of arf^educational program in- 
variably involves an 'assessment of one or more aspects of student performancje.* 

Like school cost analysis, the measurement of educational outputs in- 
volves several difficult problems* At the outset .there often p.s dis^agreement » 
about th^ specific outcome desired from an educational system and the rela- 
tive importance of 'various outcomes. Some people beJLieve every high school 
graduate should possess a saleable skill; others are primarily concerned that <k 
graduates be qualified to enter the college of their choice; still others are 
concerned that the schoo-1 inculcate students with certain values and behavioral 
patterns. , * * ^ 

^ Educational systems are expected to serve multiple and* of ten conflicting 

goals and objectives. Educational systems, for example, are urged, to provide 
students with equal educational opportunities and, at the same time, to pro- 
vide these learning experiences in the most, efficient manner possible- The 
dual objectives of equality, and efficient often conflict with each other 
and usually invol^' some type of trade-off, ^Ejtucational organizations often 
are confronted with choices that involve greater equality at the expense of 
efficiency or greater efficiency at? the expense of -^ua^ityl 

While the efficiency criterion is typically stressed in cost-effectiveness 
analysis, the ^equality aspect canrlot be ignored. Assume, for example, that 
two reading programs result in equal average gains in stident achievement* In 
the less costly program utilizing computer technology, all^ students- demonsti^ated 
about the same amounf^of gain in test scores. Irl the more costly program"^ ' , 
stressing individual tutoring, however, studehts in thie upper twc- thirds of • » 
the group tegistered modest gaiA^^^n test stores while stua.ents in the bottom 
third exhibited ^hstantial improvement. Based solely on efficiency measures, 
the first .progreon'^is preferable;^ to the%second. But if greater emphasrs is 
placed on equality of outcoi^^Y: second prpgram is mofe^ desireible than 
the first even though it^ is mdl^ costly. ^ > 

Levin (1975, pp. 114-15) poiated out that the distributional consequences 
, of a program should be considered in cost-effectiveness studies. In other 
words, he is concerned about who receiyeJ^ (or who is supposed to receive) the 
benefits of a program^ Since a'.partiGular school program will rarjely affect 
all student populations, e.g., -ittwu achievers and high achievers, in an iden- 
tical manner, an attempt should be made to examine 'Changes in the distribu- ' 
tion of giins as well as overall test score^ga-ins . . The distributional^ aspects 
of tjie gains associated with a school program clearly assume great importance 
when equalization objedtives are the major 'concern, -^-^^iss 
Techniques have been developed for treating multiple outc^ifiCs- in Cost- 
" effectiveness studies. Generally, a single *criterion of effectiveness cannot 
adequately detect and estimate the pos;5ible effects of a program. In fact, 
even the measurement of progress toward attainment of a single objective'^ > 
5f ten will require the use of multiple indicators. tK^ asse^smlent of program 
effectiveness is therefore, typically based on a set of indicators or measurer 
ments. The -selection of specific indicators of effectiveness, of course, 
depends on the objectives involved and the programs or program activp-ties linder 
analysis. ^ - . 



.An output structure similar to the cost input structure ^discussed 
earlier is needed. This multidimensional structure sjioxild include a hier- 
archical classification scheme that systematically relates programs and opto- 
gram Activities to designated "objectives. Likewise, the agreed upon goal(s) 
must be differentiated into ge|ieral objectives, performance objectives, and 
targeted performance objectives. In short, the objectives must be opera- - ' 
tiohalized and stated in measurable termfe* Several illustrative structures** 
have been devised -that delineate programs, objectives, and putput indicators ^ 
(Mushkin and Cleaveland, 1968) , 

^ince some:^stem- goals invariably are regarded as more important thafti 
others by *the relevant^ decision makers, the designated program objectives 
generally are rank ordered or prioi^itized. Typically, a weighting scheme is. 
used .to establish the r*elative importance of different program objectives. 
In using such a scheme, various weights or values are assigned to the program 
objectives or outputs based on their relative importance as* perceived by 
those persons^ responsible for decisions. This listing of wei|^hted objec- - 
tives or dutcomes can then bfe aggregated or integrated by the analyst into 
an overall 'effectiveness index. , Levin (1975) suggested the simultaneous 
use of alternative weighting schemes in order to clarify the value* j'udgments 
involved and to reveal the cost-ef fpctiveness implications of the value 
choit:es made. An explicit weighting sfeheihe has cons'iderable utility in com- 
paring multiple program outputs. 

* Similarly, program processes (activities or elements) can be rank 
ordered with regardrto their contribution to the achievement of objectives. 
By determining the relative contributions of the various program processes 
to program objectives, comparisops can be made between the different' act ivi- 
'ties within a program or across sevefiral programs designed to accomplish the 
same objectives. Just as objectives must be expressed in measurable terms) 
petfo25pance criteria should be* specified for each, program component. 

In addition to the analytical problems encountered as a re^luLt of mul- 
tiple objectives, educational programs also produce "spillover" or "side" 
effects. A program designed to improve reading achievement for example, may, 
influence pupil performance in other areas that make use ot reading skills, 
such as history or social studies. Likewise, ---a f:eaching strategy designed 
to stimulate student motivation or enhance student self-concept may also pro- 
duce gains in cognitive areas. Some programs may generate greater indirect ' ' 
effects than others. Consideration of these indirect effects, both positive 
and negative, should 6e included in an evaluation. Thus, program outc^nes 
should be measured along several dimensions. - \ 

A major problem in educational program evaluation is the selection of 
Valid and reliable instrioments with which to measure pupil performance. For 
analytical purposes, student behavioral outcomes generally can be divided into 
two categories—cognitive and noncognitive. While a distinction can be dtawn 
between tljese two categories^ they are not mdtually exclusive; they are, in 
fa9t, ^very much interrelated" Affective factors such "as student motivation, 
attitudes, and self-concept may h^ve an important effect on the learning 
process and academic achievement. However, despite general agreement that • 
affective growth is important, there has been little progi;ess in developing 
instruments 'to as§ess •affective behavior. Noncognitive objectives remain . * 
extremely difficult to define and operationalize. " " ' 
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Evaluative research in educatro^' has been basically concerned with ex- 
plaining cognitive achievement. It has largely ignored the affective dimen- 
sion of student growth. Numerous standardized tests have been developed and 
jaan^ of them can be used with some degree of confidence; Nevertheless, the 
use of standardized tests to measure cognitive achievement can be hazardous. 
The results of standardized tests are reported iivjiKyariety of^-Majf^^- and' the 
different scoring modes must ibe used in an' appropi;iate manner. 

^Almost all standardized tests involve the conversion of raw scores into 
normative scores to indicate a student's relative position in a distribution 
of scores. Grade- equivalent scores, for" example, indicate the grade level at 
which students. are performing. While grade-equivalent scores have some 
utility, Coleman and I^rweit (1970) clearly point out that these scores can- 
not be used to make inferences about the effect of a school program on the 
rates of growth of pupils who start at different grade levels. 

Specifically, a grade-equivalent score reports a student's position ra- 
diative .to' the median of the norm group at a particular grade level.^^ Accord- 
ingly, a year of growth is based on the total distribution of scores, at that 
grade^level. Since variance in test scores will increase progressively from 
earlTy'^to later grade levels, a gain of twelve months will be increasingly 
more difficult to obtain. In other words, a student who maintains the same 
percentile over time, that is, the same position relative to other students, 
will appear to fall farther and farther behind based on grade-equivalent 
measures. 

* A "year of growth" in reading at grade 12 is less, relative 

to the total distribution of 12th-graders' scores, than a year 
of grpwth at grade 6^; A "grade-equivalent" score, therefore, • 
means a different thing at every grade-level. It does not com- 
pare the* student to others^ of the same age or at the same grade 
\evel; it compares him to the average or median student at another 
gr^e level. It is a relative sc6re masquerading 'as an absolute 
. sdMe [Coleman & Karweit, 1970, 'p. 17>. 

Although percentile scores are' bet'ter- suited for comparison than are 
grade-equivalent scores, Cqleman and Karweit also demonstr^ed the mi'sleading 
nature o*f these scores. TSie percentile identifies the point In a distribu- 
tion of seores^plow which a given percentage of students fall. If a student 
scores at the 80th percentile, for example, then 80 percent of the total group 
have scores less than or equal ^o his. In the normal distribution, depicted, by 
a bett^shaped curve, student scopes are clustered closely around the 50th per- 
centile and are- much more widely, spaced at the extremes, *e*g. , the 10th or 90th 
percentile. Thus, "the percentile score stretches out the scale toward the 
middle,, and '-compresses it at the ends [Coleman & Karweit, 1970, p. 17]." Per- 
centile scor^v..>fhich permit a comparison of the relative position of students 
at different joints in tmer therefore, are useful in determining the direction 
of change but '^^-^ n<ft "satisfactory for measurincf the amount of change that has 
occurred. ' \ * " v. 

A more accurate estimate of the cimount of change can be made by using 
standardized scores. The distribution- of/standard scores* (Z scores) ia. base^ . 
on a mean of zero and»a standard- deviation of one. The relative position of 
a score is expressed in standard deviation units. The standardized score 
permits a comparison of a student's performance on one measurement with, his 

' . . ^ 
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performance on another measurement. .Coleman and Karweit (1970, jpp. 17-21) 
suggested the creation of standardized scores for. every grade lA/el. By 
standardizing, the scores, the mean would J^ecome the .same but a separate 
standard scorfe would be expressed for each grade level. The amount of .change 
could then db compared more accurately by using the same mean and standard 
deviation across grade levels. ^ ' 

V/hiie standardized tests describe* a student's position relative to other / 
students, they do nc^t diagnose the specific skills that have been, mastered, by 
the student. For this reison, criterion-referenced tests (as opposed to norm- 
referenced tests) appear to be 'better suited for use in cost-effectiveness 
studies. The distinguishing, feature of a criterion-referenced test is its 
relationship to the specific goals and subject mattei^of a^program of ins.truc- 
tion. , / . " . » 

Each item on a criterioo^refetenced test is designed to measure 
or indicate the accomplishment of a particular skill. The im- 
portant factor is which items ate passed, not the number. Test * * 
scores are for advancing the student, not generally to summarize 
achievement (Averch, et a.1., 1972, p. 33] . ' ' 

___^Like standardized tests, criterion-referenced tests ^re not without prob- 
lems.' Criterion-ref^r^'nced tests ajre developed to evaluate progress toward 
specific program objectives. ^As mentioned earlier, widespread agreement on 
specific objectives is often difficult to achieve* In addition, some objec- 
tives are 'difficult to opera tionalize in measurable terms. For these reasons, 
course objectives may be oversimplified in order to construct the necessary 
test. items. ' Other drawbacks to criterion-referenced tests are that they ^ 
generally are costly t6 ^develop and can seldom be used over a wide range 
of applications. ^ ^ • » ^ 
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CONCEPTUAJ. WORK 



COST-EFFECTIVP^ESS LITERATURE 

. .4 



In a paper basically concerned with the performance of mi4-itdry,'^systems<, 
^uade (1971) presented a conceptual framework ^or conducting cost^f f ectiveness 
analyses consisting of five basic elements — obiective (s) > alternative (s.)% costs, 
mod$l(s), and a criterion (see Figure 3). • Quad^ viewed the analytical process 
as involving three overlapping stages* In Ihe first, the formulation stage, 
thf problem is defined, the issues are clarified, and the study is limited. In 
4the second, the search atate, alternatives are generated and data are collected. 
In the final st^ge, the ^complex process of cqmpa^ring cor evaluating, the various 
alternatives is undertaken. ^* 
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Figure 3. Quade's structure of analysis. 

Source: Qvaade, 1971, p'. 296. 



^ ytiade emphasized the importance of usin^ a model, to compare alternative 
courses of a'ction in*te^ o^^ their costs and* effectiveness,. The model, e.g., 
computer simlalation, linear programming, or mathematical equations, provides 
"a precise structure and terminSlogy that serve primarily as an effective 
'means, of communication, enabling the participants in* the st-udy to exercise 
their judgment and. intuition in^a concrete context and *iri proper relation to 
others (1971, .p. 295]." While the controlled 'rjeplicated experimental approach 



can rarely, if ev^i^ J^e used in policy analysis, Quade argued that every 
effort must be made to* maintain the rigor of scientific methods. In other 
words, the alternatives must be examined systematically and objectively to 
permit others to replicate, and verify thfe study. 

Working with educational systems, Alkin (1970^ constructed a model- to 

(1) fcompare the cost-effectiveness of alternative instructional programs, 

(2) * evaluate the cost-effectiveness of specific school .programs, and (3) deter 
mine the cost-effectiveness of usiilg different school input option^. Th^li^r 
prKDgosed model (see Figure 4) consists of* five components: (1) student y 
inputs — the abilities and characteristics of the students entering th^ school 
program, {2)' f iQ^ncial- inputs — the financial resources made available to - 
support the program, (3) mani^ulat^le characteristics — the resourcfe-consvuning 

/aspects of the program that*car> be changed administratively, (4) outcomes — the 
cognitive and noncognitive changes that occur in students after they have i 
been exposed to the instructional program, and .(5) external systems — the 
social, political, legal, and economic structure of society, . « 
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Figure. 4. lAlkin's cost-ef fjectiveness model, 
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Source:' Alkin, 1970, p, 226, 



Depending , upon the type^of evaluation conducted, Alkin arranges thes^-^ 
different .components into coptrol, predictor, or Criterion variable sets. 
The co^t-ef fectiyeness of alternative instructiqnal programs, • for. example / 
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could be compared by using components and B (external systems and student 
inputs) as cohtrol variable sets, component C (financial inputs) as a predic- 
tor variable set, and component E (outc^pmesj as a criterion variable set. 
Likewise, indil^'ldual school programs could be examined by using' components A, 
B, and C as. predictor variables with component- E as the criterion measure. 
Various input' combinations could be compared by using components A, B,.ar\d 
C as. control variables, componen^t D (manipulatable characteristics) as pre- 
dictor variables, and component E as the criterion.* ^ 

. Abt. (1969) developed^a cost-effectiveness model to evaluate .ElemfeniSry 
and Secondary Education Act (ESE!A) Title I progr'^s for the disadvantaged. ^ 
Tlie model attempts to evaluate the relative school, student, and community 
effects and associated costs of alternative Title I programs. Abt's input- 
output model (see Figure 5) includes five submodels: (1) ' ^chool~the 
production procea/s of transforming the inputs (different student types and 
educational resources-) into better educated individuals, (2) instructional 
process — the changes in student behavior resulting from Title I programs, 
(3) "community — the ^impact on the community of clianges in educational outputs 
due to Title I programs, (4) costs—the direct and indirect cos^s required to 
implement Title I programs, and (5) cost-ef fectiveness~the analysis of the 
effects and f^sults of Title I programs. 
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Figure 5. Abt*s cofet-ef fectiyeness model. 
Source: Abt, 1969, p. 67. 
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This model is designed to program data descriptive of educational systems 
for computer simulation purposes. Inputs include community characteristics, 
student demogr'aphic data, and Titfe I program data. .Outputs include changes- 
in stiide^ achievement and attitude,' as well as changes in the number of school 
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graduates and dropouts. Abt developed each of J:he five submodels in consider- 
able (Jetail.. Trfe school submoctel/ for ^^xample, includes a school flow matrix 
for monitoring student achie^eitjent^nd ^^^^^stematic procedure for performing 
truancy /dr. opout calculations/,^; - . ^ 

Haggart -and Carpenter (190>9) developed a model for school district plsin- 
ning based on a plalining-prograinming-budgeting^ system (PPBS),. The model 
illustrates the various PPBS c'om^onents and clarifies the important;^ analytical 
dimension (see Figure 6)". The Pi?BS process begins with* a statement of objec-. y 
tives ar^S^^a^-categorization of underlying activities and programs. This cate- 
gorization provides the program structure (Pi, P2\ to Pn) necessary for''' ^ 
generating possible program alternatives (Al, A2, t:o An). Each^^lternative 
must be considered in terms of resource requirements, and in re^tion to several 
effectiveness measures. The evaluation and selection of a f5ref erred alternative 
fhay involve many difficult decisions. Clearly resou^e requirements will hav^e 
to be weighed against resource availability and estimates of effectiveness will 
have to be weighed against designated objectives. ^ 
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Figure 6, ^Haggart and Carpenter^s analytical model. 

Source: Haggart and Carpenter^ 1969^ p. 2, 
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Kim and Harris (1976) developed a cost-effectiveness modej. and data in- 
struments applicable to the management of secondary vocational education pro- 
grams (see^Figure 7). Eight elements for any cost-effectiveness analysis were, 
identified from the literature: (1) the program or alternative programs, 
(2) program^, objectives, (3) program cost, (4) program output, (5) a'^TBodel^rof 
th'e relationships among the elements,' (6) elff ectiveness — the extent tp which 
the objectivers are achieved,' (7) efficiency— the relationship between t^he out- 
put and the cost, and (8) the ratio of program effectiveness to program cost. 
The -model includes four major components, — vocational program classifications, 
program objectives and specifications, ^program otitputs/ and costs. It was 
designed to generate three kinds of cost-effectiveness measures — ef f e^iveness, . 
efficiency, and cost-effectiveness ratio and/or performance ratio. 
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Figure 7, Kim and Harris' cost-effectiveness analysis model for secondary 
vocational programs. • 

Source*: Kim and Harris, 1976, "p. 34^ 



Seilei- (19§9) suggested several' preliminary considerations which must be 
^^^■^^^^^ before attempting to_ meas\ire a system^s cost-effectiveness. Possible 
program alternatives should be assessed initially to determine thpse which are 
feasible within the context of a system* s ovepall liirii tat ions. In-order to 
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condupt 'a meaningful analysis^ J;be><f easible alternatives must either be : 
homogeneous or equalized with respect to many factors, such as geographical 
location or scale of measurement/ that might otherwise prevent meaningful 
con\pcirisons^ Seiler delineated five basic cost-effectiveness a:pproaC:hes — 
ratio model, indifference^ curve inodelr mathematical programming, theory of 
gajii6S/ and probabilistic cost-effectiveness* <> 

The most tomroon technique, the ratio model, measures the \"ef f iciency" 
of a System in temts of the ratio of its output (ef f ectivenessK to its j.nput 
(cost). An overall "figure pf merit" is obtained by dividing tne System 
effectiveness Mndex by the system c^st in^e^c^ A more useful approach, the 
indifference curve model, permits an analysi's of variou? combinations of al- 
ternative systems. By employing marginal analysis ,while holding cost and 
effectiveness constant, an indifference, curve model can be used to identify 
that combihation of alternatives wKich maximizes the figure of merit. The third 
technique/ mathematical programming, .includes linear, nonlinear, *and integer^ 
programitvirfg , the calc.ulus of variations, and optimal control theory. Assuming 
that the functional relationships among the related .variables are linecir and 
additive, for example, typi'cal linear programming technig[ues can be used to 
deter^mine the optimuih allocation of a system's resources or "to determine a 
system ''s minimum cost requirements. - 

In the first three models it is assumed that the decision maker can operate 
in a'^ vacuum ,^i.e., ^s^jj^ the^re were no other decision makers in his fieiS. This 
assumption, however, cannot be made in a competitive' system where the decision 
maker must consider carefully his. opponents ' possible reactions to the decisions 
that he mi^ht make. An attempt therefore, must be made to identify optimal 
strategies in confrontation situations. Assiiming eq^al costs, therlt are dif- 
ferent "payoffs" (effectiveness levels), important to .different people, over 
the- range of available strategies (alternatives). 

Ir^ the last approach, ^probabilistic cost-effectiveness, cost atnd effec- 
tiveness probability distributions are generated for* each alternative. This 
method permits, the decision maker to weigh his confidence ip the various alter- 
native Systems. In concluding his discussion of these models,; Seiler pointed 
but that external factors iijust be considered ^carefully when making a choice, 
based gn a cost-effectiveness criterion.^ Exogenous factors — technological 
advances, the ava^ilability of resQurtes, or political considerations — may sig- 
nificantly influence a syst^ and strongly affect any cost^ef f ectiveness 
decision. . * ' • 

EMPIRICAL WORK ' ^ _ - 

Levin (1970b) applied cost-effectiveness techniques 'in an analysis of 
teachej; recruitment and retenti'bn pplicies.. Since teachier salaries typically 
account for aborut 75 percent of a school's operating bydget. Levin posed ^wo 
•questions: (1> Which teacher characteristics show a relation to a goal that 
most of us would accefpt for the schools, that is, .performance on a standardized 
test of verbal achievement? and* (2) What does i^t cost the schools to obtain 
teachers with different characteristics? ' ' . * 

To answer these' questions, Levin""used data from the Coleman study to in^ "* 
vestigiltte the impact of the teacherVs verbal ability and experience on the 
performancfe of^sixth grade students on a standardized aQhieve|nent test. Using 
a production function approach,, Levin estimated »that thb effect of *each addi- 
tional unit of £eacher verbal score raised the^ verbal scores of white students 
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by an average of .179 p6ints and the .verbal, scores of black students by an 
average of .175 points. At the same time, each additional year of'fteach^r 
experience was associated with an average incj-ease of .P60 points for white 
students^'and .108 for black students.' 

, Focusing next on the cost dimension, Levii\e?camined the relationship « 

•between teacher salaries and the teacher 's. veri>a5wMlity'. and experience. 
Based on reg'ression analysis.. Levin estimated thar^eachers were, on^the 
average, receiving about $79 more for each additional year *of^ teaching ex- 
perience and about ' $24 , more fir each additional point of verbal score. Bring- 
ing the cost data and achievement resullls together. Levin concluded that 

--firing teachers with higher verbal ability would be five to ten times more 
effective per dollar of expenditure in increasing student achievement scotes 
than would hiring teachers with more experience. 

^Jamison, Suppes, and Butler (1970) investigated the potential role of ^ ' 
Computer Assisted Instructio|n . (CAI) for -compensatory education programs in 
urban schools by reviewing /he ^results of an ESEA Title III funded program in ' ^ 
CAI in New York City. The ^arithmetic achievement, scores of students receiving W 
CAI (experimental gjroup) were ' compared with those of students receiving 
traditional instruction (control group). Cost-effectiveness values were cal- 
culated for both the experimental, and control groups by combining cost and 

■ performance data under three different sets of asstunptions which specified ' - 

' the median case, best case, and worst ^case. After estimating the "costs of 
one month's achievement gain wi.th CAr for each of the three' cases, the re- ^ 
searchers concluded that t^he N6w ^rH^ CAJS program ,in elementary arithmetic 
was a hig^y cost-effective coir^e^ktpry education techhiomie. ' . ' 

Curtis (1971) Contended that^^ decision maker must-.uitimately choose 
between* program processes <ar componeivts) "ratdier than ,|ptal programs^ Accorc^-' 

' ingiy, a cosfc-effectiveness^^^odolcpy was developed t<?> (1) identify the 
processes (taskf*\7hich cpnc^j^ used, tcr attain * 

common objectives, (2)-d^e«}e the ^eia^ive*>ef fectiveness of ^ thQse processes) 
and (3) compare the cost-ef f^Siivei^ess relationshiF/.amdng' the jprocpsses. ' Two 

'reading projects in the. Milw^ee {Wisco»sin)\-P?fi^ic -Schools wbre na^^ to - 
demonstrate .this cost-ef fect^ness approach.^ Bbthl^eadi^g. projects -enrolled S 
comparable student populations, , sought to attain sdmilar objectives/ and 
employed common processes. . ' ' n 

A panel, of experts was asked to rank order tlfe obj^qlAves 'of the "two 
reading pi;ojects in terms of their perceived importahceX^^ various pro- 
cesses were then rank orderfed on the basis of their \p4rc9jr^i4^cmttibution 
to the attainment of each objective. Initially ,^ thi rese^hJj^Koaology in- 
vplved the development' of a utility value based on. the weighted objectives and 
prodesses. In the next research "procedure, three output measurQ|p~a grade 
equivalent change in reading achievejnent, the difference, between actual grade t-' 

'equivalent change and expected" change, and a change in student -^ttitude toward 
reading--were considered in dete^ining the relative effectiveness of the pro- 

' cesses. * '* t ~l 

, Curtis calculated a utility/cost valije for each profess by dividing the 

jutilifcy value by the cost pdr pupil 'for the process. Tfte relative*" contribu- - 

. tion (if each pJrocess to the overall ef f&<ft:iveoess scbre wa% determined by 
dividing the atUity value of ^ach process by the sum o^ all utility values ' * 

for ^11 proc ess^ . (The total student change score based oil all three output , , , 
meas,Ures~the«^rall effectiveness score—was ''2. 74) . I'hfPf ectiveness value 
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f6r each process was obtained by' multiplying the overall effectiveness score 
of 2.74 by the proportion of contribution made by each process. To obtain 

cost-effectiveness^ ratio for each process/ the effectiveness value for the ^ 
process Vas divide^ by its 'cost per pupil. Table 1 displays the methodology 
Curtis employed to determine cost-effectiveness of different program components. 
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The cost-effectiveness .ratios* in Table 1 give the decision maker an esti- 
mate of the amount of gain that can be associated with a dollar of investment 
in each process. Processes J (teaching aides orgcmize and catalog reading 
materials) / K (reading research teacher works with individual teachers in the 
preparation and use pf materials) , B (teaching( aides work in individual class-' 
rooms witl^^eachers in th^ e pr eparation of *roaterial§^t* and A (resource teacher 
visits each teacher's ^classroom to provide' rea<Jing infebruction suJ)po.rt) have 
the largest payoffs/ yielding approximately .04 to .Q7 tor each foliar invested. 
Process G (reading resource teacher conducts inserviCT sessions for the total 
staff) has the next largest return, yielding a gain of .02. In short, Curtis* 
methodology produced a prioritized listing of the array of possible processes 
based on the {IX importance of the objedtAves, (2) contribution of the various 
processes to the attainment of the objectives, (3) utility value of tl^e pro- 
cesses, (4) costs of the processes, and (^) ef fecfiveness of the pi^oc^sses^. 

iciesling (1972) estimated the relationship of selected edubat'ional inputs 
and the readin^^ performance of disadvantaged children yn ESEA Title I projects 
in California. Kiesling's data were based on a 6 percent, sample^ of Title I ^ 
projects in Caiifornia* The projects sample<^j.ncl\;ded 10 percent of the state's 
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,1 students enrolled ix^ the second, third, fourth, and fifth grades* 
esling found that the followingT variables best explained the pooled reading 
achievement for all students: (1) beginning score, (2) program^ length,,^ 
(3) percentage of minority group pupils, (4) minutes of instruction by reading 
st)ecialists, (5) minutes of instruction by paraprof essionals helping regular 
classroom teachers, (6) percentage of instruction in a separate facility, and 
(7) hours of plaaning per week* In a separate analysis of students in the 
third grade, however*, only beginning score, program length, and specialist in- 
struction were related to reading gains at high probability levels. The school 
input — specialist instruction—was mbst cons^-stently related to reading gains, 
witii the strength of the relationship particularly strong in the analysis of 
the third grade. Translating these findings into cost terms, Kiesling estimated 
that a $100 expenditure per pupil for reading specialists would provide an 
additional one-tenth ^f ^ain per month of instruction. An .additional expendi- 
ture of $3*00 per pilWrrir'would buy a "normal" learning rate of 0.;? months gain ' 
for participating Title I children. ^ ^ 

In another stujfty involving reading curricula, Webster (1972) examined 
the "cost-effectiveness relationship between instructional costs and student 
achievement ga^-ns in several reading program's in Grand Rapids, Michigan. The 
reading programs studied included three programs in which performance contracts 
were used: (1) Alpha II, (2) WestingAouse Learning Corporation (WLC) , and 
(3) Combined Motivj^tion and Educational Systems (CMES) , plus three additional 
programs: (4) Project Read, (5) a traditional remedial reading (TRR) program, 
and (6) a regular (control) program. Webster's data were based on a sample 
of Title I studentSL. enrolled in these p'rograms in second, third, seventh, 
eighth, and ointh grades. Because of differences in the time allotted- for , 
reading activities and unequal class sizS's, instructional costs were reduced 
to a common, denominator of student minutes of exposure (SME) for several cosi: — 
categories. Webster calculated an annual cost per pupil for each program. She 
then divided this cost by the average one-tenth (.1) grade^ gain .to establish a 
relationship between program costs and gains. ' * 

Analysis of variance was used to examine the gains in student reading 
achievement. Separate analyses were conducted for the second and third grades, 
for the seventh grade, and for the eighth and ninth grades. The mean achieve- . 
ment gains by programs and grade levels and the cost per one-tenth (.1) achieve- 
ment gain for each analysis are presented in Table 2. For the second and 
third grades, WLC, Project Read, and tfte cOttg^ol program were most cost- 
effective, Ejroducing the largest gains for the least cost. The WLC program * 
* was the least expensive per one-tenth grade gain; the TRR program was/associated 
with the highest overall mean gain but clearly was the most costly program. 
For the seventh grade, CMES and Alpha II were ipost cost-effective while TRR 
had the' highest cost per-tenth of achievement gain. For the eighth, and ninth 
grades the two performance contract programs were considerably more cost- 
effective than the control group, Webster concluded that educators should 
give consideration to whether or not traditional reading programs should be 
continued in thei?^ present form. ^ . ^ 

^ Jamison, Suppes, and Wells (1974) surveyed the research concerning the^ 
effectiveness of the following. instructional media: traditional classroom 
instruction (TP) , instructioijal radio (IR) , instructional television* (ITV) , 
programmed instruction (PI) , ^nd computer-assisted \nstructioji (CAI) . ^ 
StU(idnt ^achievement scores* were used mbst frequently to assess the eff ectit^,e- 
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TABLE 2 

WEBSTER'S COST-f EFFECTIVENESS ANALYSIS 
OF six READING PPOGRAflS' 



Reading Treatment Program 
and Meaui Achievement Gain 

Alpha II 
Westinghouse 
Project Read 

Tradition^ Remedial Reading 
Cojnbined Motivation and Education 
Cof^lrol (iegular School) 



.4 
.6 
.7 
.8 



.5 

.7 
.8 



Grades 

7 . 



.5 



- .2 
.7 
' .4 



1.0 



1.1 
.3 



:8 



.6 
.4 



J- 



Cost Per One-Tenth (.1) Student 
Achievement Gain in Reacdo^g 



Elementary Grades 2 .and 3 



Alpha II 

Westinghouse Learning 
Project Read 

Traditional Remedial Reading 
Control (Regular Program) 



22 
11 
16 
49 
14 



51 
82 
.34 

.eo 

.28 



Middle School Grade 7 



Alpha II 

Combined Motivation and Education. 
Traditional Remedial Reading 
Control 




Alpha II 

Combined Motivation and .Education 
Control 



19 
1 
27 
23 



89 



.82 



10 
13 
25 



.47 
.18 
.24 



ness of thefee alternative media. After reviewing the studies investigating TI^ 
the authors concluded that few variables appear to^ be .consistently^ associated 
with student ^performance. Two exceptions, however, were noted. First, in most 
studies, the. teacher's verbal ability was positively correlated with student 
learning. Second, while the research surveyed indicated that tSie teacher- 
student ratio has little influence on student achievement; small classes seemed 
to improve the performance^ of young children. 

Based on a limited nxamber^of sfvdies, the authors also concluded €hat IR, 
supplemented with appropriate printed materials, is a§out as effective as*TI. - 
At the same time, they fbiand strong evidence to indicate thajt ITV, usedoin a 
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manner tl«t '^simulates TI^ is as effective as TI. Likewisfi^^-wost research ^ 
findings sugg€?&t^d that there were no significant .differences between either 
PI or CAI an'd traditional classroom instruction. Some, of th^ studies, how- 
'ever; reported a saving in student time, which has important' impLications for 
overall effectiveness. There is also considerable evidence to suggest that 
the ''Use of small amounts of CAI in conjunction with regulat qlassroom instruc- 
tion leads to increased achievement, particularly for slower students. 

Jamisori and his colleagues cited the .high costs of schooling as today's 
numb^f one problem in education. The prices of school inputs, especially 
teacher salaries, have been increasing sharply witfiout a corresponding increase 
in school productivity. The ajathors believe the 'productivity of . educational 
systems can be improved by augmenting human effort with technology^ In the ^ 
studies they surveyed, however, alternative technologies were employed pri- 
marily to improve instructional quality*^ or to provide enrichment activities.^ 
The authors argue 4:hat the time has come ts? explore "much more systematically^ 
the potential of technology to reduce systfem cos^ through productivity im- 
-provement [1974, p. 58)." 

Blaschke and Sweeney .(1974) reported preliminary results of a study of 
the cost-effectiveness of compensatory reading programs in Michigan. Their 
study focused on 25 high-achieving districts and 23 low-achieving districts 
classified on the basis of 1972-73 Title I evaluation reports of approximately 
500 local education agencies. Individual schools were classified as high- 
er low-achieving on the basis of the school's -average grade equr\7alent unit^ 
gain score divided by the number of months between the pre- ctnd post'^t^sts. 
The effectiveness model employed in the study was based upon groupings ''rt;>f 
variables which the researchers hypothesized would act together in their 
impact on the effectiveness of compensatory educa)tion programs. The COST-ED 
model developed by Education Turnkey Systems was. used in the analysis of cost, 
with each program viewed as consisting of one activity in which the student was 
directly involved ^classroom reading activities) and four* supportive activ- 
ities in which tlie student -was not directly involved / (planning, training, 
decision making, and administration). The results of the study indicated 
Jtha^t "some schools in Michigan do make a difference and' that the characteris- 
tircs of these schools are in many instance's .very significant . . . and that the 
factors which appear to'describe, if not malce, the dif^rence between effec- 
tive and non-effective comp-ed programs are for the most part, 'controllable' by 
Ibcal district staff, and usually those^at th^ building level. . . ♦ [1974, 
p. 61]." Among the signific^t factors identified were the (1) role of i:he 
school. principal', including how the principal allocated time and delegated 
dyisjion making to teachers,'' (2) role? of tea*chers, inclucTing amoxint of deci- 
sion making delegated to the -teacher and amount of time allocated to instruc- 
tional management activities by the teacher, '(3) nature and extent of coprdi- 
nation among. the tejaching staff, especially between regular teachers an^t 
compensatory education teachers, and (4> .^amount of planning time allocated 
by the compensatory education director anat^chers. ' The -researchers noted 
that, with regard to resource utilization, "the factors variables which ^i. 
characterize the^jjif fererfces cost few additional marginal dollars; rather 
they reflect different time usage pattern^ of building sta'ff [1974, pp.^ 62- * 
6>]." \ W 

The^Michiganr^st-ef fectiveness study is being continued by the Michigan 
Department of Education. ^ It utilizes data for individual pupils and class- 
rooms in an atten^t^o identify and test the replicability of practices and . , 
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processes related significantly to the success of reading programs (Michigan . ^ 
Department of Education, 1976). The preliminary results, based upon applica- 
tion o€ path analysis to a substantial amount of data, indicate that 
(1) higher per pupil program costs are associated with higher reading Ichieve-* 
ment and (2) tocher morale, degree of acc9unt^bilityy and involvement , of 
paraprofe4pionals are the only variables shown -^to h^ve direct impact on 
reading achievement. , 

The Pennsylvania Department of Education is engaged in a, multi-year 
study of the quality and cost-effectiveness of special. education programs in 
that state and has reported the results obtained from analyses of the first, 
year's data (Pennsylvania Department of Education, 1976), The study <^ involves 
a random sample of srpecial education classes selected to represent five cate- 
gories of exceptional it y~educable mentally retarded^ trainable mentally re- 
tarded, socially and emotionally disturbed, brain injured-, and physically 
handicapped.' The sample was -stratified by (1) number of, pupils per exception- 
ality, (2) elementary or secondary program, (3) range of cost in the programs ^ 
and (4) demographip conditions (inner-city, other tietropoli tan area, suburban, 
and rural). In addition t© data on five cost categories (general administra- 
tion, spetial education administration, direct Instructional* costs, iijstruc- " 
tional support, and instructional materials aijd equipment) an indicator of' 
quality instrument was developed based on criteria of effectiveness deemed' im- 
portant by teachers r siipervi*Sors, parents, manbers of advocacy g±oups, teacher 
trainers, and special educators. * ' 

Analysis of the first year's 'data revealed that the expenditures for ^ 
special education did not correlate consistently with quality ^of instructional *' 
programs as me^asured by' the indicator of quality instrument. It also was 
found that, while ^expenditure for special education did not correlate consis- 
tently with achievement gains, a number of significant relationships did exist. 
It was found, for example, tl\at "cost contributed significantly to achievement * 
gains in reading and spelling for the, elementary edu^ble mentally retarded; 
in reading for the se^condary educable ment;aJ.ly retarded; in reading for tlie 
elementary trainable mentally retarded; ai^d in spelling for the' elementary 
socially and emotionally disturbed, physically handicapped^ and brain injui^ed 
(Pennsylvania Department of Education, 1976, iv]." 

Wolfe (1976) utilized data obtained from the individual* records of 627 
students enrolled in the Philadelphia Public School System over ^ three-year 
period (1967/68-1970/71) , and cost information based on t&e school year 1975/76, 
to analyze the cost-effectiveness of various ways of reducing school expendi- 
tures : Utilizing the production function relationship between school inputs 
and outputs based on the pupil data- and the 1975/76 cost infoJ^jaation for the 
scAopl input's, Wolfe examined howresources might be reallocated more effi- 
ciently if the current budget level were maintained and hbw resourpes might bfe ' 
allocated if an across-the-bdaf-d budget reduction of $30 per*pupil were reqtii 
Based on the results of her analysis*, Wolfe observed that: 

*(1) The systematic evaluation o*f inputs in relation to outputs, com- 
bined with cost figures, can increase ^the effectiveness of educationa 
^ dollairs. ... (2) The cost p^r pupil is not directly tied to ^pupil 
achievement growth. Current resources can be. used more effectively^ b;^ 
reallocation. Different expenditure patterns yield very different re- 
sults. (3) A systematic budget cut yields more satisfactory results 
than an across-the-board cut. (4) Even in times of ^budget cuts it may 
be best to increase expenditures, on certain resources' (i. e. , .Smaller 




classes for low achievers, more experienced teachers for" high 
achievers) and compensate* with larger cuts* elsewhere to maximize 
the out^put of the school system [1976, pj)/ 18^19] \ 




56 



> 



VIII 

A, CONCEPTUAL FRAMEWORK FORJEgONOMIC ANALYSIS OP SCHOOLING 



Educat;Lonal organizations and the learning process are • extremely com- 
plex. At^ present, educational decision makers have little jnore than intuitioA 
to guide them in determining how to most efficiently us^ the resources avail- 
able to schools. There is little reason to expect greateA productivity in 
educational systems until more i inforfnation concerning the impact of various 
types of resources on school learning is available to guide educational . 
decisioa makers. * ^ • 

' A systems model of the educational production process (see Figure 8) 
can provide a heuristic framework for analyzing and thinking about the 
'problems inherent in maximizing the productivity of educational resources.- 
The conceptual framework dirscussed in this: section views the educational^ ^pro- 
duction process as a system subject to economic analysis. We take the posi- 
tion that school resource allocation is primarily an economic problem and 
that the tools of economics and systems analysis should be- applied to ques- 
tions concerning the efficient utilization of school resources. The' model ^ 
we have otulined consists of four major components: (1) inputs to the 
educational system, including policies which constrain and/or control the 
systfem's operation, (2) the formal educational system (school) and the 
processes associated with that system, (3)' outputs df the educational system, 
and (4) feedback. The conceptual framework, shown in Figure 8," is equally 
useful for viewing a classroom, school, se^aool district, or even a state as 
an educational production systemr The framework follows the resources which 
are provided to the formal educational system from its external .environment 
(the school coi(imunity, school district, state, or nation), through the ^ , 
educational process which occxirs within the school, to the ediicational out- 
comes; The feedback compon^n^ ties system outputs to \>oth th^ educational 
pr'ocess arid the system inputs. Changes can be made to modify either the 
process or inputs, in or4er to ;ivore efficiently accomplish the objectives. 

System environment and control ^ . ' 

Inputs to the educational system from its external environment can sig- 
nificantly influence the outcomes of schooling. Figure 9 shows in gre'ater 
detail this first' major component of the concepttial framework. Re^search 
cited earlier in this paper clearly indicates that a ccxnmunity's socio- 
econoinic characteristics, values, attitudes> and expectations bear a signi- 
ficant relationship to the outcomes of schooling. Whilq most people agree 
that the primary function of schools is "to educate," their views ^as to 
what constitutes education Vary widely. Various subpublics often *ho;Ld dif- ' 
ferent expeqtations for "the schools. ♦ * 
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gpy^Q Figure 8, A conceptual framework of* the educational produclAon process under school conditions. 
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Another significant input from the external environment is knowleag^;."- _ 
about the educational process. Professional educators are expected to ^.^\ 
possess knowledge eibout hum^n learning and to use"^is knowledge in * - 
selecting appropriate learning activities for students and in making effi- 
cient and effective use of the resources made available to the school ♦ 

A community's economiq base has an important bearing upon its ability 
tp finance education. The level of funding available to the school is 
strongly influenced by the economic resources (fiscal capacity) of the 
community and its willingness (teix effort) to support education. Factors 
such as househo-ld income levels, mcinufacuuring activity, and retail sales 
cire important 4eterminants of the communit;,y ' s economic base. Although the 
level of fiscal resources available to a 'school is primarily dependent upon 
the community,' state policies concerning educational finance and federal 
aid programs also are imTOrtant f actors. _ 

The social and— desSfttSgraphic characteristics of a community constitute a" 
third set of' variables that influence the educational production process. 
Variables such as the educational level, age, and occupatidn of adults inter- 
act with economic and community factors to shape attitudes and expectations I 
Papulation growth or decline and the age structure of the population also 
affect 'the resource input of the school. It must be borne in mind that a* 
school does not exist in a vacuum; it exists in' an identifiable milieu. The 
educational processes of the school inevitcibly-must reflect the nature ^of 
the community served,' the ne^ds of the pupils in attendance, and the ex- ^ 
pectations held for the*school by parents and the public. • 

System Controls 

Public schools must opergit^ within a well defined policy framework. 
Aims, priorities, .and controls are established for schools 'by elected repre- 
sentatives at local, state, and federal government levels. "An extensive 
system of constitutional requirements, judicial mandates, statutor^p.direc^ 
tives, and administrative rules either influence or control the educational 
production process. 

Schools depend primarily on local property taxes and state and federal ^ 
aid .prograitTs for their operating revenues. In addition to the controls built 
int;^ its general school aid formula, a state may exercise control over educa- 
il policy by imposing .spending limitations^ establishing minimum educa- 
il standards, prescribing curricula, stipulating certification requirements 
professional personnel, ^d the like. ♦ 

At the- local level, a community supplements the controls established by * 
the state by formulating its own set of rules. and r^gulatipns through a ^ 
board of education* This board determines (either explicitly or implicitly) 
the relative importance of various goal^s,. establishes priorities, ctnd iden- 
tifies^ objectives. Contracts between a board of education and a teacher or- 
ganization can impose constraints upon class size, working hours, length of 
the school 'year, provisions fqr inservice improvement, compensation, and other * 
variables directly retated to the educational production process. 

The aims established for a school district must be translated into edu- 
cational objectives and defined in terms t^^^an operating educational program. 
The relative importance ^assigned to the vaAous objectives will help' estab- 
lish priorities for* use of available resources. 

• « 
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THE EDUCATIONAL SYSTEM . ^ 

The second major component of the conceptual framework is the educationa 
system itself. This may be further subdivided into two elements — school re- 
source inputs and school resource applications. A school district or school 

^ has two general categories of inputs with which to work — human^resources and 
material resources. Because the set of aims and priorities reflecjtefi in %he , 
system controls affects how these resources may be cUsed, school personnel at 

. all levels, from the classroom to the central office, have certain, const^iiints 
within^, which they must operate as they seek to achieve specif ic learning" ob- 
jectives. ' ^ ' 

School Resource Inpttts ^ / 

m ^ ' ■ ^ — 

Resource inputs to the school may be grouped into two major categories, 
as shown in Figure 10 — human ^resources (i.e., students, teachers, and staff), 
and material resources (the physical plant, classroom equipment, curricular 
matei^ials, supplies, and the like) . While school administrators cannot con- 
trol important home or community background factors or student characteristics, 
they can exercise control 'over some school resource inputs and the way they 
are deployed 'to help overcome factors that can negatively affect student 
learning. ^ ^ ^ ^ . o • * ^ 

Students are the most significant input to the sphool from its external 
^ environment. Research hds shown that home and cpmmunity background factors 
- will strongly influence their aspirations, motivations', skills, and ki^owledge. 
Unlike a manufacturing plant which can reject raw*Vnaterial that does not 
meet quality standards', public schools must work "^with the pupils* who attend 
them. They cannot arbitrarily reject students who faiJ!l|to meet some pre- 
-established admission requirement. Because of the differences which exist 
*^ among students, teachers and administrators must be thoroughly familiar 

with the communi,ty served by the school and musif tailor tme educational pro- 
cess to meet the ne^ds of individual students. For examile, disadvantaged 
students are likely to need different educational programs and experiences ' 
than advantaged sti^Qents if they are to achieve their i^^ll potential. 

The personnel employed by. the school — teachers, administrators, guidance 
counselors,^ psychologists, social workers, librarians, teacher aides, and 
other personnel directly involved in the learning process — constitute an 
important inpyt. Similarly, personnel not directly involved in teaching/ 
such as maintenance Workers, bus drivers, and food service" workers, are im-" . 
portant^ Research ^cited previously h^s demonstrated that teacher characteris- 
tics are significantly related to some school outputs and -that certain attri- 
bute''^ .of teachers bear significantly upon some learners and not upon others/ 
'Thus, a school administrator must carefully consider^ the characteristics cpf 
current staff members when assessing the qualifications of prospective staff 
^meinbeiis in order to identify the candidate who will best me^t^ th^ needs of 
the particular studepts to be served/ In addition, the school administrator 
must consider competing priorities, for example, .balancing the need for addi- 
tional classroom teachers against the need for specialists* in counseling or 
health' service areas^ ' ^ / 
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Figure 10. Resource inputs to the school. 
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The largest material resoifece, of course/ is the school physical^ plant — 
the buildings and grqunds. In addition to the school plant r'-t^omputer terminals, 
audiovisual equipment, desks, b&oks, and an extensive array of other learning 
aids and equipmer\t are utilized by students and teachers -in the course of 
the educational production prpcess. Previous research has not consistently 
revealed significant relationships between student achievement and variables ^ 
such as the age' of the school building, the percentage df makeshift -class- 
rooms, or number of' library volumes, although these variables have, occasionally 
been identified as signif io,ant . ' Because education is a .highly labour intensive 
activity^, significant cost saving^ ^eventually may be achieved thorough applica- 
tions of innovative technolc^gy. A computer-managed instructional program, for 
example, might permit^ a reduction in cost and, at the same, time, increase ' 
teacher productivity by performing tedious record-keeping functions. "Para- 
professionals or iteacher aides might be able to perform non^teaching duties 
which would otherwise.be performed by. a teacher* . ^ ' 



The Resource Input Mix (Program Alternatives) ' , . ' 

A major task of the school administ;:ator and professijDnal jstaff is' 
identifying the most appropriate manner in which hiaman and material resources 
may be combined to achieve the goals* and objectives Qf the "school , effectively 
and efficiently within the constraints imposed b^ the system' s /control policy. 
In other words, school administrators and teachers must transform the school 
resources at their disposal, into educational prograios. Figure 11 illustrates 
some of the variables they must consider. ' , - 

School administrators and teachers must rely upon their knowledge and, 
(^training in organizing the most appropriate instructional programs and in 
^electing relevant learning activities and experiences for students. Since 
knowledge about human, learning is accumulating "rapidly, teacher^ and admin- ' 
istrators must constantly keep abrdast of new information gained through . 
research and incorporate it into operational instructional programs. In 
determining the most effective resource mix,^ the instructional content — 
reading, mathematics, laiJguage arts, science,., etc. — as well as the instruq- 
tional process variables of the educational program must be taken into 
account. In attempting to achieve performance objectives established for 
specific /curriculum programs, a school staff must make several iipportant deci- 
sions concerning resourc^e use. What type of student grouping patterns will 
best facilitate the learning process — independent study, one-to-one, small 
groups of three- five students, class size groups of 25-30 students, or larger 
groups? How can stQdent time best be utilized? How can the necessary sup- 
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portl^Je services best accoiranociate the. programs? what type of curriculum 
materials should be^ used? _ ^ 

Of particular importance in the instructional process is the utiliza- 
tion of time. In Carroll's model of school learning (1963), time is a. central ^ 
va^riable. The model's thesis is that students differ in the amdunt of time S 
they need to master a given unit of learning to a set criterion. As Bloom ' 
has noted, "All l.earning,* whether done in school or elsewhere, requires ^ 
time. . . . Time for school .learning is even more limited by the resources 
available for it, by the ways in which these resources are mcide available. 
to particular segments of the population, and by the ways in which schools' 
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and individuals use the time available to them [Bloom, 1974, p. 682].". 
Wiley and Harnischfeger (1974) pointy out that several aspects of time as 
a variable are subject to policy mani^lation^ for example, length of the 
schoor year and s^ool day. They also pointed out that, while the length 
of the' school year and school day establish the maximum time available for 
exposure to school instruction, "within the limitations imposed by thi^ 
maximal amount 9f time,, the actual exposure of a pupil to instruction is 
determined by his attendance, the instr^ictional programs, and the^location 
decisions which occur within jihe classroom [1974^ p. 11]."' Thus, time, must 

bj considered^ a signif icarrTtr^actor in theVresource input mix-, s 1^^^- ^ 

/ Unlike socioeconomic factors or the innate abilities of students 
professional educators can manipulate educational program and process vari- 
ables in an effort to enhance 'studertt'' learning. Unfortunately, they do not 
always exercise this discretion wisely' as evidenced, for example, by the 
common .practice of Yielding to *a uniform class size. It) wdlTld appear from 
the research that class, size^ should depend more on interrelationships among 
teacher characteristics, curricular areas, and student abilities, than upon " 
administrative convenience. ' ^ 

Educational decision makers need to generate an^s examine various resource 
mixes. - Cost-effectiveness comparisons become possible when "value-added" 
measures (e.g., gain scores) and the cos^ts associated ^ith them are obtained 
for eaj^h program alternative. The relative cost of all inputs, as well as 
vtheir impact on the learning of indiviidual students, must be analyzed care- 
fully in order to determine how 1^ allocate resources more efficiently. For 
example, some students may learn more in smaller classes but a sizeable 
across-the-board reduction in class size may not be feasible because of the 
cost involved. Thus, decision makers may wish to seek out other policy 
changes which'may produce siirvilar results at considerably lower costs %nd 
hence greater e'fficiency. * . 

^OUTPUTS' OF THE EDUCATltlflAL SYSTEM - 

The third major ofomponent c5f the concfeptual framework, illustrated in 
Figure 12, encompagsefe the outputs of the educational system. As was the . 
case in the other coiftponents, value preferences come .into play. Th&re 
exists considerable disagreement about the objectives of schooling and the 
priorities which should be assigned to the various outputs -of schooling. 
Even within a school-district or the attendance area served by a school, 
Aibstantial disagreement may exist among various subpublics on questions 
concerning the goals and priorities of schooling. 7 

The outputs .of educational system may be clas/ified in various' w^s. 
For. example, outputs can be categorized as shdrt-^rafige and Ipng-rang^., as 
cognitive and af fective,' t>r as -monetary and nonmonetary. Thd fact that th^se 
categories are not mutually exclusive provides potential for a very Wmplex 
matrix, of outputs. HoweVe/ categorized, outputs mijst be compared wi«i the 
goals and objiectives established ^or the. educational, system, .ap well as with 
program cost. The system's effectiveness and efficiency are evaluated by 
comparing outcomes and costs with es^iablished. goals arid objiectives. This 
performance evalpaiwLon identifies discrepancies between objectives and re-, 
suits, and provides the system with information which may be used 'to vali-* 
date or modify either the inputs, the process, or both. <> ^ 
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^-^^ purposes of illustration, we have identified fdve categories which 
might ^e used to classify the outguts of schooling — short- and long-range 
outputs/ monetary and nonmonetary outputs , and joint outputs. Note that 

, these are not mutually exclusive categories • Subject matter, mastery may be 

*' classified as eithej a short-range or nonmonetary output^ or H^th, ^ Sindlarly, 
income may be classified as a long-range output or as a monetary^ output. 

*".The purpose^ of '"the analysis is the determining factor in choosing the measures 
of output that should be employed, - O 

A number of input-output studies have focused on the monetary outputs of 
the Educational system utilizing rate of return analysis to evaluate the 
private and/or. social rates of return 'to investments in educaJtion at various 
levels of schooling, e.g., completion of eight grades^ com^^letion of twelve 
grades, completiqm of four years of college,, etc. The analyses have focused 
'large groups of individuals rather than^oif individual students as investors. 
Although researchers irecogftize that both the* private and the public sectors 
of the economy benefit fropi ^vestment in education, it has >proven difficult . ' 
to forecast the returns to individual students from such an, investment . 

The shprt-range outcomes of the educational process ma^J^ be demonstrated ^ 
in many ways.^ Perhaps the most familiar are standardized measures of cognitive, 
affective, and/or psychomotor performance. However, students may establish^ 
that they have accomplished educational objectives by demonstrating. their 
possession of basic knowledge; displaying intellectual oi^ motor skills; dis-, 
playing powers of reasoning and criticism; demonstyMtincf' through behavior and 

^ performance the possession of certain values, att;itSles, and motivation; 
expressing through their actions a sense of cultural appreciation or a. sense 
of social responsibility; or demonstrating their ability to learn independently. 
Some outcomes of schooling cart be ascertained through standardized achievement 
tests or tests of basic knowledge; other outcomes are best assessed by ob- 
serving a student'' s performance of certain tasks requiring intellectual and/jor 
motor* skills. Still others are best assessed through anecdotal records and 
observations of students both within* and outside the school. .It is imperative 
^that there be a direct* connection between the objectives^ established for the 
school and the performance measures used to assess educatiori^^ outputs. If 
schools are to be held accountable for the performance of certain functions", 
the measures by which they JDbe judged^ must accurately reflect the* established 

• objectives. This implies that measures in addition to performance on 
standardized tests> mu»t be utilized if t^ie outputs of the educational pro- 
cess are to be Evaluated adequately -and JEair^y* 

Other outcomes which merit corffe iteration are \^at we have termed "joint 
outputs." Joint outputs of the educatiqijal process are those which occur 
whether or not they are sought and* whi€fF*hideed may be uriiijtended. For example, 
a possible joint outcome of the educational process is a change in staff 
morale. Chang^s in the system inevitably v^ll affect the morale of tecRThers, 
^ ^dhm u^trators , and others. Although chants in staff morale A^^e not often 
a primary objective of the educational process, virtually any change in, the 
system has potential for affecting staff morale. Joint outcomes of the 
educational process are analogous to the smoke produced by* an industrial 
plant, or the odor produced by a paper mill J It is not the primary objec- 
tive^ of. a factory or |nill to produce smoke or odor, yet these j.ncidental 
(and often unwanted) outputs may be of great concern. Any analyst who seeks 
to understand the educational production ptocess must ^e ^ware that joint 
outcomes are likely to be produced and should be sensitive to the potential 
significance of such outcomes. ' *^ . % 
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FEEDBACK 



Th^ fourth major component of the conceptual framework '{>ortrayed in 
Figure 8 is the feedback loop. The fe^dback component is the system's self- 
correcting mechanism. Feedback^ is produced by comparing the system's outputs 
with its objectives*' It should be noted that feedback occurs continuously, 
whether or not it is planned by the scV>ol* For example, if the graduates of 
a school cannot read as weXl as they should, parents, employers, and interested 
citizens will promptly inform the school and expect corrective actions to be 
taken. It is through planned evaluation of the system's, performance (i*e, , 
compa^ring outputs and objectives^ that resource allocations can most effectively 
be altered or modified to achieve a better match between objectives and results • 

Feedback can provide a basis for altering the allocation of resources 
witHin the educational system, itself or it qan result in modifying the resources 
made available to 'the system from the external environment. Dissatisfaction 
with the output of the system may, fpr example, result in a decision to make 
more (or less) resources available to a school.' . Similar ly^ feedback may result 
in decisions which alter the nature of the instructional process within a given 
curricular area by instituting changes in time allocation, grouping procedures^ 
or staffing patterns. Feedback can also alter the aims and priorities established 
for the system or the controls established to monitor system perfo'rmance. Thus, 
the feedback component ties the system together and ensures that it remains ^ 
dynamic. 



IX 



IMPLICATIONS FOR RESEARCH ON 
COST-EFFECTIVENESS IN EDDCATTON 



^ - Although few* would disagree with the statement that the educational pro- 

' cess is extraordinarily complex, the implications of this stat^ent for research 
on cost~effectiveness-in education have not yet been fully realized. Our know- , 
ledge of ^ the learning p;:ocess is sketchy. Although there are various theories 
of learning, none has been validated suf f iciei^^tly to serve as a reliable basis 
for prediction. The interaction of the wide variety of variables which bear 
upon the outcomes of the learning process is not well understood'. In fact, ^- 
even if it were possible to conduct controlled experiments, we do. not yet know 
which variables are most important to control. Much of the research to date 
has utilized macro-measures of input and output and has focused at the 'school 
district, state, or national level. Future research must focus on the indivi- 
' dual' pupil and should be longitudinal in design. .Unless future research focuses, 
on individual pupils, it will be difficult or impossible to answer questions 
about what instructional procedures, materials, and processes are tjest for 
whom and under what conditions. ' . ■ ' / 

Laboratory experiments are neither feasible nor practical in cost-effectiveness 
research. Even if coi>trolled experiments could be performed, the crucial question 
for educational deci^pn makers is "What happens under school conditions?" It 
must be recognized that it is difficult to assure either randomness of subjects 
or precision of treatments when data are gathered^under school conditions. Thus, 
researchers mu6t be reconciled to the fact that data o;i school input, process, ' 
and output will always be somewhat "dirty."" , ' • ^ 

It is also important that attention be directed to those variables which 
are amenable to control by teachers and administrators, t^hile it is important 
and useful to know that certain socioeconomic- background variables may bear 
heavily upon a child's performance "in school, such variables are generally 
beyond the control of the school. Variables within the corftrol of administrators - 
and teachers offer more promise in terms of improving cost-effectiveness rela- 
tionships in education. • . . 

The problems of^cquiring data on both cpst and process variables merit 
further consideration. Disaggregated data concerning the monetary costs of 
various school inputs are virtually nonexistent. Few school systems ar§ able 
to provide data on the cost of. operation of individual schools, much less/on 
the fiscal inputs to various curricular programs within a school, it is even, ^ 
difficult to^ obtain from educational personnel accxirate estimates of the time 
they spend on various tasks. Furthermore, the cost of Conducting time and 
motion studies is prohibitive. The task of obtaining daXa concerning the • 
attrirutes of individual pupij.s has been complicated by recent federal and' 
state legislation restricting access, to such information, while the^^objectives 
of tne legislation are laudable, it-^oes impede the progress of investigators' - 
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interested in conducting cost-effectiveness research ♦ Although the projplems 
involved in obtaining datd on individual pupils are not insurmountable, the 
time and cost involved in obtaining such data "have been increased substan- 
tially. . . ' . ^ 

.The problems of obtaining data on school inputs are relatively simple, 
however, when compared to the problems associated -with measuring outputs. 
Perhaps the roost perplexing problem is that general agreement does not exist 
concerning the* priorities which are to be assigned to* specific educational ob- 
jectives. For example, whiLe nearly everyone agrees that schools should turn 
out "good citizens," it is extraordinarily difficult '"to obtain consensus on 
any operational measure of a "good citizen." And eyen if we could agree on 
feuch measures, by the time data are collected and analyzed the educational^ ^ 
process is likely to have changed during the interim*^ 

Research on costr-ef fectiveness conducted on specific curricular progiiams 
and products and on •alternative educational processes^sing short-run measures 
of output holds promise^ for increasing productivity, ^owever, alternative 
programs and processes can validly be compared only when their specified ob- 
jectives are very similar. Since specific (as opposed to general) educational 
objectives should be established for units- no larger than a^schaol district, 
and prefe^rably no larger than an individual school, cost^ef fectiveness research 
should focus atM:hese, not at state or national levels. 

Although the'^ difficulties involved in conducting research on school pro- 
ductivity and efficiency should not be iinder estimated, these difficulties 
should not deter researchers. Research on productivity and efficiency is a* 
logical and necessary -next step in^he' continuing search fqr equality of edu- 
cationai- opportunity.. Only through research adding to our knowledge, of the 
interrelationships and interactions between variaBles affecting educational 
outcomes can we hope to achieve greater eqviality of outcomes for students. 
Scholars in school finance are now able to design schqol finance J)rograms - 
V^hich Jwill -assure equality of fiscal inputs ^to school districts and even to 
individual schools. These ^technical solutions admittedly are not always , 
politically feasible but the knowledge necesisary to develop such solutions 
exists- Attention must now be directed to achieving greater equality in the 
outcomes of the educational process. -^Economic aijalysis of the educational 
production sy«tem offers a valuable tobl to help abhieve this end. 
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